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THE ROUTINE COMPUTATION OF PARTIAL AND 
MULTIPLE CORRELATION 


RAYMOND FRANZEN AND MAHEW DERRYBERRY 


American Child Health Association, New York City 


Probably the simplest and most direct method of eomputing 
multiple correlation and the regression coefficients of a multiple 
regression equation is the formation of a system of normal equations 
from the moments and product moments and the solution of these 
equations by the Doolittle method as reported by Tolley and Ezekiel 
in Vol. XVIII of the Journal of the American Statistical Association. 
A modified form of this method to be used when the intercorrelations 
are known and the raw moments are not available was published by 
Garrett.* It often happens, however, that not only are the regression 
coefficients and the multiple correlation desired, but the partial 
correlations are needed for interpretative purposes. In such problems 
it is more advantageous to compute the multiple correlation from the 
partial correlations according to the usual formula: ! 


Risa... n = 1 — (1 — via) (1 — r%iss) (1 — reas) ~~ + (1 — r%nes... ni) 
and the regression coefficients from the formula: 





Die.s4...n = 

nv (1 — 7719)(1 — r713.2)(1 — 714.23) - - > (1 — r'tm.23.. . n—1) 
"or (1 — r7y2)(1 — r7o3.1)(1 — regis) +> (1 — r?ancas. . . 21) 
But the computation of ry.234 . . . »a-1 and the other higher order 


partials becomes increasingly laborious as the number of secondary 
subscripts are increased. It can be shown that the formula 


n+2(n—1)+3(n—2)+-°---n(n—n-1) (1) 


* A Modification of Tolley and Ezekiel’s Method of Handling Multiple Corre- 
lation Problems. Journal of Educational Psychology, January, 1928. 
641 
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gives the minimum number of partials that are necessary for computing 
a partial of the nth order; or, in other words, it gives the minimum 
number of times the usual formula 


. an T12 — T3723 
we V1 _ risV/1 — 195 


must be applied to finally obtain a partial of the nth order. It will be 
noted that formula (1) gives the minimum number of computations. 
The possible number that may be computed is many more than that 
given by formula (1). For example, to compute a fourth order partial 
it is possible to compute thirty-five separate partials before finally 
obtaining a fourth order partial. On the other hand, only twenty 
partials need be computed if-:the most economical combinations of 
correlations are made in the computation of the lower order coeffi- 
cients. Even though the computations are made in such a way that 
the minimum number of pzeriis!s are computed, the usual method still 
requires considerable attention to avoid an incorrect combination in 
one of the lower order partials which would make all the subsequent 
work erroneous. 

The method here presented completely eliminates these difficulties 
by reducing the solution to mechanica! routine. It claims the follow- 
ing advantages: 

1. The nth order partial correlation is obtained in the minimum 
number of operations, since the possibility of duplication of computa- 
tions is eliminated. 

2. Correct combinations of lower order partials are routinely made 
assuring the accurate higher order partial. 

3. All computations are automatically checked without a double 
computation as is often advised. 

4. Repeated writing of figures is eliminated. 

5. Because of the direct mechanical and systematic calculation all 
the computations from first order correlations to the multiple can be 
done by a clerk. 

6. A compact, concise record of the computations is provided for 
filing so that all partials of a given order and denomination are kept 
on the same page. | 

7. The multiple correlation is obtained from the partial forms with 
a minimum amount of computation. 

The explanation below may seem difficult and complicated, but 
the routine is simple and easily followed once the method is under- 
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Computation of Partial and Multiple Correlation 643 
Dracram I 

riz 

Tis * Tas 

Tis — Tis 2s 

kis + kas 

Tis.s 

Tis T23 

Tis * Tas T25 * T35 

Tis — Tislas Tes — TasT as 

kis + kas kes - kas 

pa Tass 

Ti4 P24 Ts 

Tis * Tas Tos * T 45 T35 ° Tas 

Tia — TisTas T24 — TosTas Tsa — TasT as 

kis + kas kes + kas Kas - kas 

Tis.s Teas Tacs 

T15 T25 Tas T4s 

kis kes ks kas 
(Check 1) (Check 2) (Check 3) 

Tos + Tas + Tos Tas + 135 Tas 
(Check 4) (Check 5) (Check 6) 
Kas + has + kas | kas + kas kas 

ria + ris + ris Tos + 12% T 34 

(ris) (Check 1) (res) (Check 2) (rss) (Check 3) 

> Numerator = Numerator > Numerator 





| (kis) (Check 4) 





(kes) (Check 5) 





(kss) (Check 6) 


















































DiacraM II 
T12.5 
T13.5 23.5 
T14.6 724.5 34.5 
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stood. The computational form with the entries and checks in terms 
of symbols is given in Diagram I. Here the operations necessary to 
compute a set of first order partials from the interrelationship of five 
variables are indicated. Diagram II denotes the method of trans- 
ferring these first order partials to a second form for the computation 
of second order partials. 

To make the diagrams more concrete the multiple correlation of 
four bony measurements with weight and two regression coefficients 
have been computed from the table of intercorrelations given below, 
and each step in the computation explained. The use of Diagrams I 
and II with the explanation will reduce the seeming complexity of the 
detailed directions that follow. 


TaBLE I.—INTERCORRELATIONS OF ANTHROPOMETRIC TRAITS IN ONE THOUSAND 
ELEVEN-YEAR-OLD GIRLS* 








Weight Height Chest breadth} Chest depth 
ASR ene .7428 
Chest breadth...... . 8337 .6298 
Chest depth........ .8201 . 5388 .6845 
Hip width......... . 8752 . 7334 . 7323 .6956 

















* Data gathered in connection with the School Health Study conducted by the 
American Child Health Association where this routine was devised and used. 


STEPS IN THE COMPUTATION 


1. To compute first order partials the r’s are entered on specially 
ruled sheets (see Diagram III) just as they would be entered in an 
inter-r table with the exception that four lines are left below each r for 
the entry of computational figures. The order in which the coefficients 
are entered determines the order in which the variables will be par- 
tialled out, and before beginning computation this order must be 
decided upon so that the r’s will be so arranged on the sheet as to take 
care of this demand. The correlations of all other traits with the 
variable to be partialled out first are entered as the last row of inter- 
correlations, the correlations of all other traits with the variable to be 
partialled out second is placed second from the bottom, etc. Thus in 
this computation we have chosen to partial out hip width from all the 
remaining inter-r’s first and chest breadth second. (Compare Table I 
and Diagram III bite: arrangement of variables.) 


-~ ~*~,  ~_F,hUlUCOr, 
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2. Immediately below each of the correlations in the last line (16) 
of intercorrelations (the correlations with the variable to be partialled 
out first) is written the k or +~/1 — r? of the coefficient appearing 
directly above it (line 17). ‘These values of +/1 — r? are taken from 
“Tables of «1/1 — r? and 1 — r?” by Miner. 


Driacram III* 


Weight 
Height (1) . 7428 
(2) 
(3) 
(4) II 
(5) Height 
Chest Depth (6) .8201 . 5388 
(7) 
(8) 
(9) III 
(10) Chest Depth 
Chest Breadth |(11) . 8337 .6298 .6845 
(12) 
(13) 
(14) IV 
(15) Chest Breadth 
Hip Width (16) . 8752 . 7334 .6956 . 7323 
k (17) . 483761 .679797 . 718429 . 680982 
(18) 
(19) 2.1613 1.4279 . 7323 
(20) 2.079208 1.399411 . 680982 
(21) 2.3966 1.1686 .6845 
(22) 
(23) 


* To simplify the explanation in the text the lines and columns are numbered. 
The numbers are not necessary in the ordinary computational procedure. 

























































































The computations described in Steps 3, 4 and 5 below are used in 
the check and are not necessary for the computation of the partials. 

3. Beginning at the extreme right of the last row of intercorrela- 
tions (line 16) add each figure towards the left recording each cumula- 
tive total three lines below (line 19), and in the column corresponding 
to the last figure used in the cumulative total. Thus, .7323 is written 
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in the first column; .7323 + .6956 is written in the second column, 
and 1.4279 + .7334 is written in the third, and so on towards the left 
until all of the r’s have been added except the r in the last column to 
the left. It is not necessary to include this r in the summation. 

4. The next line (line 20) is obtained from the line of k’s (line 17) 
beginning similarly with the figure at the extreme right, and adding 
each figure in succession with the exception of the k in the last column 
to the left. 

5. Sum each column of inter-r’s omitting the last line of r’s (line 16) 
and record on line 21. For example, 2.3966 is the sum of .7428 + 
8201 + .8337. 

Computation steps 6 to 14 are illustrated on Diagram IV. 

6. If a calculating machine is used put the r that is in column I and 
in the bottom row (.8752 in row 16) in the machine and multiply by 
each succeeding rin that row. The products are written beneath each 
of the zero order r’s in column I in succession. Thus the .6419 entered 
on line 2 is .8752 X .7334; .6088 entered on line 7 under the next zero r 
in the first column is the product of .8752 X .6956 and .6409 on line 12 
is .8752 X .7323. 

7. Similarly put the r in column II and in the bottom row (.7334 
in row 16) in the machine and multiply by each succeeding r in that 
row and record the products in column II in succession on lines 7 and 
12. The entry in column III, line 12, is the product of the r in column 
ITI on the last line (.6956) and the r in column IV (.7323). These 
computations give the encircled term of the formula below for each of 
the six partials on Diagram IV. 





Tay — PeePey 


Tey. = keekey 


8. Subtract each of these products obtained in Step 7 from the r 
just above it in the column. Thus, .1009 in line 3 is .7428 — .6419. 
(The operations indicated are, of course, always performed algebrai- 
cally, giving proper regard for signs.) This is the numerator of the 
formula given above. 

To check result thus far: 

9. Multiply the r in the bottom row of column I (line 16) by the 
figure in column II, line 19, and enter the product in column I, line 22. 
Thus, 1.8916 is the product of .8752 X 2.1613. Likewise multiply 
the r in the last row of column II by the figure in column ITI, line 19, 
and record the product in column II, line 22; 7.e., the 1.0473 is the 








CEE Tl” 
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product of .7334 X 1.4279. The remaining product in column III, 
line 22, is found in a similar manner. 
10. Subtract the products obtained in Step 9 from the summations 
immediately above them on line 21, and enter the differences in line 23. 
11. Sum the numerators in column I on lines 3, 8 and 13 obtained in 
Step 8. The sum should equal the figure shown in column I, line 23, as 


‘the result of Step 10, (7.e. .1009 + .2113 + .1928 = .5050).* Of 


course in some cases there may be a difference of 1 or 2 points in the 
last decimal place due to dropping of decimals. In a similar manner 
columns II and III are checked. 

To compute the denominators: 

12. Perform the multiplications described in Step 6 using the row of 
k’s, line 17, instead of r’s, line 16, and enter the products on the lines 
below the several numerators obtained in Step 8. This gives the 
denominator of the formula above; (i.e. .3289 = .483761 X .679797). 

To check these entries: 

13. Multiply the k in column I by the summation in column II on 
line 20. Record in column I, line 24. Sum the denominators in 
column I obtained from Step 12, lines 4, 9 and 14. The two results 
should check. Similarly multiply the k in column II by the summation 
in column III and the product should equal the sum of the products of 
k’s in column II. 

Final computations: 

14. Divide the numerators obtained in Step 8 by the respective 
denominators obtained in Step 12. The results are usually written 
in red (italics here) or otherwise made distinctive so they can be 
picked out readily. This quotient is the partial. For example, 
.1009 + .3289 gives .3068, which is the partial correlation of height 
and weight independent of the influence of width of hips. Similarly 
.3579 is the relation of chest breadth to chest depth independent of 
hip width. The only check on this division is the multiplication of 
the denominator by the quotient, obtaining the numerator. 

To compute second order partials: 

15. The first order partials obtained on Diagram IV are entered as 
shown in Diagram V. The arrangement is the same as Diagram III 
except that no entries are made on line 16, since all the correlations 





* This may be shown to be true by the following equation: 


(riz — rises) + (ris — Tishas) + (ris — TisTas) = 
Tie + 7is + ria — Tis(T2s + 735 + Tas) 
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with hip width have been partialled out. The last line is now chest 
breadth. These entries are checked by summations of the successive 
columns of Diagram IV agreeing with the summations of the corre- 
sponding column of Diagram V. By performing the computations 
described in Steps 2 to 14, the variable chest breadth will be partialled 


Diacram IV* 
















































































I 
Weight 
Height (1) .7428 
(2) .6419 
(3)  .1009 
(4) .3289 Il 
(5) . 8068 Height 
Chest D (6) .8201 5388 
(7)  .6088 5102 
(8) .2113 .0286 
(9) . 3475 . 4884 lll 
(10) .6081 0586 Chest Depth 
Chest B (11) .8337 6298 6845 
(12)  .6409 .5371 5094 
(13) .1928 .0927 1751 
(14) .3294 4629 4892 IV 
(15)  .5853 . 2008 8579 Chest Breadth 
Hip Width (16) .8752 7334 6956 .7323 
k (17)  .483761 679797 718429 680982 
(18) 
(19) 2.1613 1.4279 7323 
(20) 2.079208 1.399411 680982 
(21) 2.3966 1.1686 6845 
(22) 1.8916 1.0473 5094 
(23)  .5050 1213 1751 
(24) 1.0058 — 9513 4892 














* Diagram IV is like Diagram III with the remaining computations completed. 


out. The results obtained from these computations are all relation- 
ships of two of the variables irrespective of hip width and chest breadth. 

Each of the successive higher order partials can be obtained by 
repeating the process, partialling out one additional variable with 
each repetition. 








Height 


Chest D 


Chest B 


Height 


Chest D 
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Diacram V 
I 
Weight 
(1) .3068 
(2) .1172 
(3) .1896 
(4) .7944 II 
(5) .2387 | Height 
(6) .6081 .0586 
(7) .2094 .0717 
(8) .3986 |—.0131 ul 
(9) .7571 .9148 | Chest 
(10) .5265 |—.0143 | Depth 
(11) .5853 .2003 |.3579 
(12) .810817| .979735|.933760! 
(13) 
(14) .5582 |.3579 
(15) 1.913495). 933760 
(16) .9149 .0586 
(17) .3267 .0717 
(18) .5882 |—.0131 
(19) 1.5515 .9148 
Co|mputation| of 3rd |Order Par|tials 
(a) (b) 
I I 
Weight Weight 
2387 — 5265 
— .0075 — .0034 
2462 . 5299 
8501 .9710 
£896 main aiwd mies 
.5265 |—.0143 | Height} .2387 |—.0143 
.850175| .999898 971093} .999898 
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Computation of the second and third order partials are pictured 
in Diagram V. The third order partial (a) is obtained in the usual 
manner. The third order partial (b) is also given in this diagram 
and will be used later in the computation of the regression coefficients. 
It is obtained from a rearrangement of the second order correlations. 
This is always possible when computing partials of the order n-2 
(where n is the number of variables involved), for 








ee na (Tis.45 . . . n)(Te3.45.. . 2) 
rinse. n = (kis.as. . . n)(Kes.45 . " n) - 
ry =e 718.45... 8 (Tis.45.. . n) (123.45... n) 

cea ata (kie.as . . . n)(Kos.45.. . 2) 


To compute the multiple correlation: 
The multiple correlation coefficient may be expressed as a function 
of the alienation coefficients of the partial correlation, thus: 


Ri.234 i a Me V1 gs (K12) (Kis.2) (Ki4.23) aed? Bs (Kin.2s ‘< tad 


If the criterion variable has been entered in column I in the computa- 
tion of the successive higher order partials all of the k’s required in the 
above formula have been determined except kines... n-1. But 
Tin.23 . . . n—1 has been computed so the corresponding k can be quickly 
determined from Miner’s Tables. In the illustrative table the k of 
.2896 is .957148. By a multiplication of the k’s in the first column on 
each successive sheet, and the k of the highest order partial in column 
I the multiple k or (ki.234 . . . ») will be obtained. Thus ki.2345 = 
.483761 (Diagram IV, line 17) X .810817 (Diagram V, line 12) 
xX .850175 (Diagram V, line 7) X .957148 (Kk of .2896) = .3192. 
From this product R.234s which, of course, is ~/1 — k:i.234s is obtained 
from Miner’s Tables. In this example R:.234,; = .9477. 

To compute the regression coefficient: 

If we substitute in the multiple regression equation 


Xi = Die.sasX2 + dis.casXs + Brg.23sXa + Ois.ssaXs — C (A) 


for bi2.345, 013.245, etc. the partial r’s and the partial o’s that are their 
equivalents the equation reduces to 




















o1 k1.2345 
Xi = ris. 3455 (Kas) (ho4.s) (Ke3.48) (Kia. we 
ode! uit ky .2345 
3 (k3s)(Ks4.5) (Kes.45) (Kas. ma 
o1 k1.2345 
+ 14. 235 > (kos) (kaa. 2) (Kas. 23) (Kis. as 
o1 Ky .2345 —C (B) 


+ 115. 234 o% (kos) (K35.2) (kas.23) (Kis. ss” 


wy — ~_ ~——_—- =" 
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If, in terms of the illustrative problem, we let X,; = weight, X2 = 
height, X; = chest depth, X, = chest breadth and X; = hip width, 
then it can readily be seen that all the partial r’s and partial k’s in the 
first two terms of equation (B) are taken care of by the computations 
on Diagram IV and Diagram V. From Diagram IV we get kis, kes, 
kss, and from Diagram V we get kos.s, ksa.s, kes.4s, 712.345 ANA 713.245. 

In order to compute the partials necessary for the two remaining 
terms it is necessary to make the same number of computations as 
those made in Diagrams IV and V, reversing the order in which the 
variables are partialled out; in terms of the illustrative problem, partial 
out the variables in the order height, chest depth and chest breadth in 
addition to the order hip width, chest breadth and chest depth as 
illustrated on the diagrams presented. Such computations would 
yield the partials required for the remaining two terms of equation (B). 

In problems involving more than 5 variables it is only necessary 
to go from zero order r’s to the (n — 2) order partials one time in 
order to obtain the Multiple R. If the regression equation is desired 
it is necessary to compute four (n — 2) order partials in the manner 
indicated above and then return to the second order r’s, rearrange them 
on the work sheet and then compute other partials of order (n — 2). 
For example, if the problem involves 7 variables, and in the first 
computation the variables are partialled out in the order 7, 6, 5, 4, 3, 
giving the two partials rie.s4se7 and 713.24567, and if in the second com- 
putation the variables are partialled out in the order 2, 3, 4, 5, 6, giving 
the partials ri7.2345¢ aNd 116.23457, the remaining two partials desired 
could be obtained by partialling out in the order 2, 3, 7, 6, 4, giving 
T14.23867 ANd 115.23467. But the computations necessary to partialling 
2 and 3 out were done in the second computation, so a rearrangement 
of the sequence on the computational form of the second order r’s 
is all that is necessary for these additional higher order r’s. This 
again reduces the amount of computation necessary. 

Though the explanation may seem long and complex, the routine 
is very simple. After it is understood and has become mechanical 
in operation, the time saving is approximately one-half of the usual 
method. It can be explained to clerks verbally much more easily 
than has been possible in this written explanation. 
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A STUDY OF QUESTIONNAIRE TECHNIQUE 


FRANK K. SHUTTLEWORTH 


Yale University 


This study shows that under the conditions defined below the 
enclosure of a twenty-five cent piece in a mail questionnaire brought 
32.4 per cent more replies than the same questionnaire without the 
coin. Six hundred eight persons receiving the coin returned 51.6 per 
cent replies while three hundred seventy-six not receiving the coin 
returned only 19.1 per cent. Data are also presented on the rdle of 
selective factors and on relative costs. 

The study is a by-product of an evaluation of the Cattaraugus 
County Health Demonstration in western New York. For a period of 
seven years the Milbank Memorial Fund financed an intensive 
demonstration of rural public health work in Cattaraugus County. 
In the spring of 1930 a staff of experts under the direction of Professor 
C-E. A. Winslow! undertook to evaluate that work. The author 
assisted in a survey of the educational aspects of the demonstration 
among school children and adults. With some reluctance a question- 
naire by mail was used to assay the attitudes of adults especially 
toward the financial support of public health work. One of our chief 
doubts about the questionnaire was the fear of an inadequate propor- 
tion of replies. To solve this difficulty, it was decided to enclose a 
twenty-five cent piece secured in a coin mailing card with each ques- 
tionnaire. Evaluation of the results required a control county and for 
this purpose, Steuben, the county just east but one from Cattaraugus, 
was selected. The inquiry to these counties consisted of a short 
personal letter each of which was separately typed and obviously so, 
the questionnaire proper containing six questions, a stamped and 
addressed envelope, and the coin. 

The investment involved and the novelty of the procedure naturally 
led to an attempt to test the effect of the coin. Accordingly, the 
inquiry was duplicated in all respects save only two sentences in the 
personal letter and the presence of the coin. This inquiry was sent to 
persons in the six New York counties immediately surrounding 
Cattaraugus and Steuben. 

The personal letter which accompanied the coin was as follows: 





1 Winslow, C-E. A., ‘‘ Health on the Farm and in the Village’”’ Macmillan, 1931. 
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A responsible organization has asked me as an impartial outsider to make a 
survey of opinion toward public health work in the western counties of New 
York State. They are anxious to have 100 per cent replies to the questions and 
accordingly have provided funds which make possible the enclosed coin. It is 
not in payment for your trouble in answering the questions but rather a small 
token of appreciation of your cooperation. 


The letter not accompanied by a coin substituted the following for 
the last two sentences in the above letter: 


The results of such a survey will be trusted to the extent that every one cooper- 
ates in answering the enclosed questions. I will be grateful if you will indicate 


your opinion and send it to me quite promptly in the enclosed stamped and 
addressed envelope. 


These letters were written on the stationery of the Department of 
Education of Yale University. 

The comparability of the two areas to which these inquiries were 
sent is of importance in evaluating the influence of the coin. In con- 
nection with the health study, which was searching for differences 
between Cattaraugus and Steuben, all available population, economic, 
and educational statistics were examined. On thirty-seven basic 
factors the two counties proved to be very muchalike. Cattaraugus is, 
of course, very different from Steuben in one respect having had a well 
organized and adequately financed county health program for seven 
years while Steuben had had no organized county health program. 
However, the replies to the questionnaire showed only slender advan- 
tages on the part of the Cattaraugus respondents. For the purpose 
of this study we feel justified in combining them into one area which 
will be referred to as the coin area. Additional data testing the 
comparability of the coin area and the non-coin area was not examined 
for the reason that only the portions of the six counties most immedi- 
ately adjacent to Cattaraugus and Steuben were canvassed. Instead, 
reliance is placed on the similarity between Cattaraugus and Steuben 
which are separated by the intervening county of Allegheny. The 
complete details are recorded in the accompanying table in the order in 
which they are discussed in the text. Data for the coin area are also 
given separately for Cattaraugus and Steuben counties. 

The inquiry itself provided other controls. The questionnaires 
were sent only to names listed in village telephone directories, thus 
introducing an economic control, and approximations to social and 
cultural controls. In the non-coin area sixteen village exchanges 
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serving forty-two still smaller hamlets and cross-roads were sampled. 
In the coin area thirty village exchanges serving fifty-seven additional 
hamlets were sampled. The sixteen villages in the non-coin area 
average 8.6 miles from the borders of the coin area. All of the ques- 
tionnaires were mailed on the same day. 

Questionnaires were mailed to every eighth name on these exchanges 
excepting the names of business firms and of women. This gave 
three hundred eighty letters to the non-coin and six hundred seventeen 
to the coin area. Four and nine letters were returned for insufficient 
address. Of the three hundred seventy-six and six hundred eight 
persons who presumably received the inquiry, seventy-two or 19.1 per 
cent from the non-coin area and three hundred fourteen or 51.6 per cent 
from the coin area made some reply. The difference of 32.4 per cent is 
sixteen times as large as its probable error. 

The coin area returned 2.7 times as many replies. This ratio grad- 
ually increased throughout the period during which replies were coming 
in. The first three days with a total of one hundred fifty-three replies 
showed a ratio of 2.5, the second three days with one hundred fifteen 
replies showed a ratio of 2.6, while the second, third, and fourth to tenth 
weeks showed ratios of 2.7, 3.9, and 4.9.! 

A detailed analysis of the responses is presented since it throws light 
on two problems of the questionnaire method. One of the haunting 
ghosts of this procedure is the selection which presumably is involved in 
the fact that some persons reply and others do not. Presumably less 
selection is involved in 52 per cent replies than in 19 per cent. On 
the other hand there is the question whether the presence of the coin 
tended to disturb the answers. The data offer many tests of these 
questions although in the nature of the case it is impossible to say 
whether one or both factors are responsible for any differences. Inter- 
pretation of similarities is equally difficult since they may mean that 
neither selection nor the coin disturbed the answers or that these two 
factors tended to cancel each other. 

A preliminary point of some interest is that fourteen of the three 
hundred fourteen respondents from the coin area returned the coin. 
Seven of these filled out the questionnaire in detail and by comments or 
implication indicated that they thought the coin unnecessary while 





1 Subsequent to the preparation of this article ten weeks after mailing the 
questionnaires an additional ten replies were received up to the end of the fourth 
month. Nine of those were from the coin area and one from the non-coin area. 
These additional cases are not included in this analysis. 


th 
ar 
ou 


qt 
al 
av 
ar 


ar 








A Study of Questionnaire Technique 655 


seven returned the coin without the questionnaire or with the question- 
naire left blank. Although the questionnaire said specifically ‘“‘ You 
do not need to sign your name” twelve of the fourteen or 86 per cent 
gave their names while among the other three hundred respondents 
only fifty-two or 17 per cent gave their names. 

The sixth question in the inquiry asked whether the respondent’s 
taxes on real estate amounted to over $100 the previous year. In the 
non-coin area, of sixty-eight answering this question, 63.2 per cent 
reported more than $100 taxes. In the coin area of two hundred 
eighty-five answering this question 59.5 per cent made this report. 
The difference is 3.7 + 4.6 percent. All of the data were tabulated 
separately for those reporting more and for those reporting less than 
$100 taxes but since no differences in the answers of the two groups 
appeared they are combined in the discussion which follows. 

The first question in the inquiry asked the respondents to rank four 
functions of public health work in order of importance. Forty-eight 
from the non-coin area and one hundred fifty-eight from the coin area 
properly followed instructions in ranking the four functions. The 
percentages of correct rankings for function (a) were 47.9 and 44.3 per 
cent, for function (>) 81.3 and 77.9 per cent, for function (c) 27.1 and 
36.7 per cent, for function (d) 54.2 and 51.9 per cent. The largest 
difference here is 9.6 + 5.3 per cent. 

The second, third, fourth, and fifth questions concerned financial 
support of public health work. Favorable attitudes were expressed 
by answering ‘‘No”’ to the second question and ‘‘ Yes’’ to the third, 
fourth, and fifth. For the non-coin and coin areas the percentages of 
favorable replies were 79.4 and 77.4 per cent, 70.5 and 70.4 per cent, 
41.9 and 39.5 per cent, and 73.8 and 77.2 per cent. Here the largest 
difference is only 3.4 per cent. 

We turn next to the care with which the inquiry was answered by 
the two areas as indicated by the proportion of omitted or uncertain 
answers. Instead of seventy-two and three hundred fourteen cases as 
our base we shall use seventy-one and three hundred seven since one 
respondent from the non-coin area wrote that he had mislaid his 
questionnaire and seven from the coin area returned the coin without 
answering any questions. For the six questions the non-coin area 
averages 15.0 per cent omissions or uncertain answers while the coin 
area averages 17.0 per cent. Analysis by specific questions, however, 
reveals one significant difference: Only 32.9 per cent from the non-coin 
area failed to rank the four functions of public health as instructed in 
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comparison with 50.4 per cent from the coin area. The difference of 
17.5 per cent is 4.1 times as large as its probable error. On two other 
questions the coin area showed more omissions while on the other three 
it showed fewer omissions. The largest of these differences is only 
6.2 per cent. 

The respondents from the two areas were equally ready to back 
their replies with their signatures. Although the questionnaire 
specifically said, ‘‘ You do not need to sign your name,” 15.5 per cent 
from the non-coin area signed their names. After deducting the 
fourteen cases who returned the coin and naturally wanted to receive 
credit for it there remain 18.0 per cent of the respondents from the 
coin area who gave their names. 

A final test of the influence of selection and of the coin is the interest 
indicated by comments. These were invited by ‘‘ Use the reverse side 
for comments”’ following the sixth question. From the non-coin area 
44.5 per cent made comments while from the coin area only 33.2 per 
cent made any comments. The difference of 11.3 per cent is only 2.6 
times as large as its probable error. While a smaller proportion of 
respondents from the coin area made comments, those who did so 
made more and longer comments. Counting each qualification to a 
specific question and each paragraph of discussion as a comment, the 
non-coin area averages 1.78 comments per person commenting and 
.79 comments per person replying. In the coin area these figures are 
1.92 and .64. Counting all words in the comments gives an average 
of 30.40 per person commenting and of 13.50 per person replying in the 
non-coin area in comparison with 48.92 and 16.25 in the coin area. 
The most significant of these differences is only 2.2 times as large as its 
probable error. The differences are also exaggerated because these 
measures are the ones in which Cattaraugus approaches significant 
superiority over Steuben. 

The relative costs of the two procedures are of some interest. No 
expense was spared to give the inquiry a favorable appearance. The 
letters to the non-coin area cost 10.4 cents each while those to the coin 
area cost 36.5 cents. The initial expense to the coin area was 250 
per cent greater per letter. The larger proportion of replies reduced 
this discrepancy very much. In costs per reply of any kind, the coin 
area exceeded the non-coin area by only 31 per cent. In costs per 
relatively complete reply the excess for the coin area was only 32 per 
cent. In costs per reply containing one or more comments the excess 
was 80 per cent. In costs per comment the excess was 61 per cent. 
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In costs per word of comment the coin area exceeded the non-coin area 


by only 12 per cent. 


SUMMARY 


The enclosure of a twenty-five cent piece in a simple questionnaire 
sent out by mail brought 51.6 per cent replies while the same question- 


naire without the coin returned only 19.1 per cent replies. 


TaBLe I.—Summary or Data 






































Non-coin Coin Steuben 
Per Per Per 
No. cent | No.| cent No.} cent 
| or M or M or M 
yr | 
Village telephone exchanges sampled.......... ge weak PSS 
Other hamlets included...................... | 42 » pte i. 2k; 
STE TS a rar pe yy | $80]...... 617 ey 
BET GUEIOD, 0 co cc cccccctdecccscve Be aes 9, re edNe 
Persons receiving inquiry.................... :; —_ RE 
FEE FT OP eT Tee 721 19.1 314! 51.6 5 
Wrote that questionnaire was mislaid......... 1: 
Returned coin with questions answered........| ... FRCP a 
Returned coin without answering questions... .. eee bia | ks ohhd 
i Adciciceauhhoed és edb ge ee cio 71, 18.7 | 307; 50.5 6 
Reported less than $100 taxes................ 25) 36.8 124) 43.5 9 
Reported more than $100 taxes............... 43| 63.2 | 161; 59.5 ia 
Correctly ranking function (a) fourth........... 23) 47.9 70| 44.3 3 
Correctly ranking function (6) first or second....; 39) 81.3 | 123) 77.9 7 
Correctly ranking function (c) third........... 13) 27.1 58) 36.7 be 
Correctly ranking function (d) first or second....| 26) 54.2 82) 51.9 3 
Correctly answering question No. 2 ““No”..... 50| 79.4 | 215) 77.4 a 
Correctly answering question No. 3 “‘ Yes"’..... 43| 70.5 | 197) 70.4 ‘7 
Correctly answering question No. 4 “Yes”.....| 26) 41.9 | 111) 39.5 8 
Correctly answering question No. 5 “ Yes"’..... 45, 73.8 | 220) 77.2 8 
Uncertain or omitted answers to six questions...| 63) 15.0 | 313) 17.0 3 
Uncertain or omitted tax report.............. 3| 4.3 29} 9.5 6 
Uncertain or omitted ranking of functions. .... 23) 32.9 | 156) 50.4 .0 
Uncertain or omitted answers to question No. 2 8| 11.4 36; 11.7 1 
Uncertain or omitted answers to question No. 3 10) 14.3 34) 11.1 1 
Uncertain or omitted answers to question No. 4 9| 12.8 33) 10.7 4 
Uncertain or omitted answers to question No.5| 10; 14.3 25) 6.i 8 
Persons signing names.....................-- 11\ 15.5 eb seed 
Deducting fourteen cases returning coin, twelve | 

of whom signed names.....................- aa os 54) 18.0 
Persons making comments................... 32) 44.5 102! 33.2 .6 
Average number of comments per person com- 

ihn 4 éas ecb ine beadaens ohne 6aeann 1.78 1.92 .61 
Average number of words per person commenting 30.40 48.92 .82 
Average number of comments per person reply- 

DtlGsinhsscaventahesabenesesedbeaen« .79 | .64 .49 
Average number of words in comments per per- 

i od sack 4d ewe ieee eee ceeeases 13.52 16.25 .19 
Total expense of sending inquiry in dollars..... 39.40 225.50 .61 
Costs per letter sent out in cents.............. 10.4 36.5 5 
Sas Ber reply of any kind in cents........... 54.7 71.8 .0 
Costs per tively complete reply in cents... . 55.5 73.5 4 
Costs per reply with any comment in cents... . 123.0 221.1 1 
Costs per comment in cents.................. 69.1 | .../115.1 52.7 
Costs per word of comment in cents........... 4.0 \ 4.5 6.2 




















Shae 














ee eee 
pay. 
Penis 


™_ 


if a aS ere a eee fae - 
po ae a 
& ote 


< ae * 


AS th # 
ee ee 
55 FREE 











3 ~ gS = 
Sia ay a 


ene ss 


> ee 


= , a. Pt ay, ee ee 
es ‘2 
~ . te ogee es ey < 


658 The Journal of Educational Psychology 


Detailed analysis of the returned questionnaires shows negligible 
difference between the non-coin and coin areas. Cn one question the 
respondents from the coin area showed significantly more omissions or 
uncertain answers, but considering omissions and uncertain answers 
to all questions the advantage of the non-coin area is only 2 per cent. 
A larger proportion of respondents from the non-coin area made 
comments, but the comments of respondents from the coin area 
tended to be longer. 

From the absence of differences in the responses of the non-coin 
areas to the questionnaire, one of the following inferences is warranted: 
Neither the selection involved in a small proportion of replicz nor the 
presence of the coin was disturbing to the replies. The influence of 
selection and of the coin cancelled each other. 

While the initial cost of enclosing a twenty-five cent piece made the 
letters to the coin area 250 per cent more expensive than letters to the 
non-coin area, the larger returns from the coin area reduced the excess 
in terms of unit costs as low as 12 per cent. At the most the actual 
excess expense is 80 per cent per unit of returns. 

The procedure of paying the victims of a questionnaire is recom- 
mended, not for adoption, but for serious consideration. In terms of 
costs it did not prove superior in this instance. Whether on the whole 
it was superior in the present case is a matter of judgment which must 
go beyond the actual data in evaluating the importance of selective 
factors and the possibility that the coin itself was a disturbing factor. 
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INFLUENCE OF THE ASSIGNMENT ON LEARNING 
DAVID H. BRIGGS 
Florida State College for Women 
A. M. JORDAN 


University of North Carolina 


The more effective utilization of the assignment as an instrument 
of instruction has been the burden of much discussion but of little 
experimentation. Recent writers such as Odell, Crawford and 
McDonald, Stone, Sears, and Monroe have pointed out the need for 
adequate assignments of lessons. By ‘adequate’ they imply that 
assignments should be clear, helpful as to study procedures, stimu- 
lating, and that they should arouse problems to be solved. With the 
outcomes of these procedures they have not concerned themselves. It 
has seemed to us worthwhile, therefore, to investigate the amount of 
added learning which accrues from certain assignment procedures. 

The assignment procedures, five in number, were chosen because 


of their general use rather than because they are the best. These 
procedures are: 


(a) The pupils were furnished a suggested study procedure which consisted of 
looking up meaning of difficult words, drawing lines under the important state- 
ments, and learning the significance of the statements previously underlined. Great 
care was taken to make sure that the pupils knew exactly what they were to do 
before they were allowed to begin. 

(b) Carefully selected questions on the material to be studied were given to the 
pupils before they began studying. The instructions were ‘‘I want you to find 
the answers to these questions, and when you have found them draw a line under 
the correct answer. Try to learn as many other facts as you can while you are 
reading and studying the selection. You are to use these questions as a guide in 
helping you to master the points brought out in the selection.” 

(c) The pupils were taught the meaning of the difficult words in the material 
to be studied. A sheet was presented to the pupils containing a list of the more 
difficult words with their appropriate meanings with these instructions: ‘After 
we have spent about ten minutes studying these hard words I am going to give 
you a test to see how many of them you have learned. Then, too, you will need 
to know their meanings in order to answer the questions you will be asked later 
on in the passage you are to read.” 

(d) Attempt was made to show the children the manner in which the tariff 
(the subject of the material studied) affected them personally. Illustrations were 
presented of types of goods made more cheaply in foreign lands than we could 
manufacture in this country because of the lower standards of living obtaining 

659 


. etme « 

ae 23: -— Fux 
it eT aes, eee: 
oo at PS eS 


th 
, 
} 
¥ 

a 

bs * 


a « 
i 


Sos SS RRR ESS 


a 


tad 
a oe bugette 


sit 


ee ee ee ae 








660 The Journal of Educational Psychology 


there than here. The ordinary high tariff argument was presented which empha- 
sized the effect of lowering or raising the tariff on wages and family income and the 
consequent effect on the pupils themselves. 

(e) In this case, there was a conscious attempt to explain what the tariff 
was, why some thought it a good measure, why it is called a protective tariff and 
why it was levied. It was emphasized that many of the pupils might differ from 
the President politically but that it was their duty to learn much about a topic 
discussed in Congress at that very time. 


The material used was selected from the message of President 
Hoover on the general subject of the tariff. It consisted of two pages 
of single spaced typewritten material and was in nature more difficult 
than the ordinary reading matter used by pupils of the levels studied. 
The four pages were divided equally into two sections, A and B. This 
could be done rather easily and naturally since there was a break in the 
thought at the point of division. For each paragraph there were 
prepared three sorts of material. The first consisted of a series of 
words chosen from the more difficult words of the text with multiple 
choice answers. The second type of material was a test of sentence 
meaning. Care was exercised in making the answers depend on the 
understanding of the individual sentence and of that alone. The third 
type consisted of statements based on the interrelation and meaning of 
several sentences taken together. Sometimes these questions depended 
for their answer upon the interpretation of what wasread. These three 
types are called: (1) Word knowledge, (2) sentence meaning, (3) para- 
graph meaning. Before the experiment proper, this material was 
checked for its difficulty (1) by counting the number of words in each 
selection more difficult than the first five thousand in ‘“Thorndike’s 
Word Book” and (2) by testing the material on two hundred forty-one 
cases located in Grades V, VII, andIX. This last procedure furnished 
us with the following results: 

















Word Sentence Paragraph 
knowledge meaning meaning 
ee os Ak ew b Ba tee 13.11 6.06 4.47 
MSRM Sia a em eg ie: 14.03 5.54 4.16 
Differences of means................. .92 .52 32 
ames Wr sb ee 37 14 12 





The PE of the differences of the means showed the differences 
between A and B in sentence meaning and in paragraph meaning to be 
reliable. The difference was unreliable in the case of word knowledge. 
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The differences between the means were provided for by presenting A 
and B in alternate manner. If there were two grades, in the one we 
used B for the control group and A for the experimental group; in the 
other, A was the control and B, the experimental. The correlation 
between the two forms was .86 + .01. 


GENERAL PROCEDURE 


The five situations were tested in the Durham and Raleigh, N. C., 
schools. Two groups of fifth, two of seventh, and two of ninth grade 
pupils were subjects for each of the experimental situations. This 
made a total of one hundred sixty-seven subjects for Assignment A, 
one hundred eighty-six for Assignment B, two hundred fifteen for 
Assignment C, one hundred eighty-nine for Assignment D, and one 
hundred ninety-four for Assignment E—nine hundred fifty-one subjects 
in all. The same examiner (Briggs) gave all the assignments and all 
the tests to each of the thirty groups of pupils. 

The general plan was to test the children first with one of the 
forms on one day, then the next day make the assignment with the 
other form and calculate the differences scored by pupils who on one 
day were influenced by certain types of assignments and on the 
other were not so influenced. The total time consumed by the pupils 
in these procedures was from thirty-five to fifty minutes. 


RESULTS 
The average results only are given on account of the lack of space. 


TasBLe I.—ReEsvtts ror AssIGNMENT A 
(Suggested Study Procedure) 


Word Sentence Paragraph 
knowledge meaning meaning 








i rok co Sune bktwane cae eee | 15.58 6.34 4.73 


«cella eo cdaescakee een | 18.66 6.43 4.80 
a elle oon, a, oe aaa | 167. 167. 167. 

Difference of means................. 3.08 .09 .07 
ESS TEE PT | .53 .23 .19 











In studying Table I, one is struck by the slight differences between 
the means. In the sentence meaning and in paragraph meaning the 
differences are merely chance differences but in word knowledge there 
was a reliable gain due to the assignment. It is to be remembered 
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that the very procedure of Assignment A almost forced an improve- 
ment in vocabulary since it consisted among other things of looking 
up the meaning of difficult words. Since we had at our disposal the 
records of several grades it was not difficult to get an average gain for 
the year in each of the tests. The average gain per year on the vocabu- 
lary test was 3.07 words which is just about the same as the improve- 
ment of the vocabulary scores as a result of this procedure. The mean 
gain per year for sentence meaning was .81 questions and for paragraph 


TaBLeE II].—ReEsvutts ror AssIGNMENT B 
(Assignment of Pertinent Study Questions) 








Word Sentence Paragraph 
knowledge | meaning meaning 
es Ck) ap ar eae eae 16.48 6.33 4.78 
IRAE read Dae pera tr tes 16.33 6.53 4.88 
Na oo a ke ee ee 186 186 186 
Difference of means................. —.15 | .20 .10 
ee i ae eee 47 21 .17 














meaning .75 questions. Although the meanings of the words were 
studied and learned sufficiently well to recognize their correct meaning 
among five words they were not learned sufficiently well in the time 
allowed for them to have entered into the warp and woof of the sentence 
in such a manner as to improve the scores on the sentences and 
paragraphs. 


TaBLeE III:—ReEsvuutts or AssIGNMENT C 
(Teaching Meaning of Difficult Words) 











Word Sentence Paragraph 
knowledge meaning meaning 
eS I AR a IR ay 15.06 6.34 4.74 
CS SS 2? 26.69 6.60 5.03 
a, ss ave kaw 6 ond 215 215 215 
Difference of means................. 11.63 . 26 .29 
Pines. OF meeeee.....<...... 212,00... 41 .20 .16 











It is clear from Table II that the assignment of pertinent study 
questions had little if any demonstrable result on word knowledge or 
paragraph meaning and a slight but unreliable effect on understanding 
the meaning of sentences. 
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It is apparent that the words learned in the experimental group did 
influence the scores in word knowledge. More interesting perhaps is 
the small improvement in sentence and paragraph meaning when so 
many of the words had been learned. The improvement looms larger, 
however, when compared with the yearly growth of .81 questions in 
the sentence meaning and of .75in paragraph meaning. The difference 
between the experimental and the control in word knowledge needs no 
comment since these words were taught the children directly. In 


TasBite IV.—ReEsvutts or AssiGNMENT D 
(Making Pupils Aware of Personal Value of Materials) 








Word Sentence Paragraph 
knowledge meaning meaning 
I dr Se et ie ee 17.87 6.96 5.43 
i BO SD ee 18.57 6.92 5.48 
EEE aes er ae 189 189 189 
Difference of means................. .70 — .04 .05 
Ne. ann cab eaeaeehs .49 .22 18 














sentence meaning the chances are sixty-two in a hundred that if the 
experiment were repeated the experimental group would still be ahead 
of thecontrolgroup. In the case of paragraph meaning similar chances 
are seventy-eight in a hundred. Teaching the meaning of difficult 
words, as far as our study goes, has a decided effect upon the ability to 
comprehend the meaning of sentences and paragraphs. 


TaBLe V.—Resvutts or AssIGNMENT E 
(Making Pupils Aware of the General Background of the Material) 








Word Sentence Paragraph 
knowledge meaning meaning 
Te i aera 17.32 6.77 5.48 
vid cx enetostentes 18.09 7.07 5.59 
EE us 5p ab ck sn ode Okaee 194 194 194 
Difference of means................. .77 .30 ll 
INN, osc kw sews ebeccsi 47 .20 16 














As a result of making pupils aware of the personal value of the 
tariff, there was a fairly significant improvement in word knowl- 


edge. The effects on sentence meaning and paragraph meaning were 
nil. 
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Making children aware of the general background of the material 
studied produces a positive effect in the three types of answer required 
of them. The differences are not statistically reliable but all of 
them are positive and in the same direction. Compare these differ- 
ences with a year’s growth of 3.08 in word knowledge, .81 in sentence 
meaning and .75 in paragraph -meaning and it is clear that a few 
minutes of explanation brings about 14 of a year’s growth in word 
knowledge, 44 of a year’s growth in sentence meaning, and 14 of a 
year’s growth in paragraph meaning. It will be seen that the 
improvement is greatest in sentence meaning and least in paragraph 
meaning. 


TaBLE VI.—MEAN GaINs IN EAcu OF THE SITUATIONS WITH THEIR PROBABLE 





ERRORS 
Word Sentence Paragraph | Num- 
knowledge meaning meaning ber 





(a) Suggested study procedure 8.08 +: .53 .09+.23} .07+.19 | 167 
(b) Pertinent study questions. 15 + .47 .20 + .21 .10 +.17 186 
(c) Teaching meaning of words] [11 .63|+ .41 .26+.20| [.20)4.16 | 215 
(d) Making pupils aware of 
personal value of materials) .70+. po —.04+.22; .05+.18 189 

(e) Developing background...!  |-77/+.4 [30+.20; .11+.16 | 194 
Average yearly gain........ 3.08 81 .77 

















Table VI summarizes the results of this investigation. It will be 
seen that the scores encircled are as large or larger than their corre- 
sponding probable errors. From the strictest statistical point of 
view only Assignments A and C are reliable for in these cases the 
differences are more than four times the probable error of the difference. 
But in these cases the teaching directly influenced the scores on Word 
knowledge. It is clear that the teaching of the meaning of words pro- 
duced an improvement in sentence meaning of .26 of a question or a 
little less than 44 of a year’s growth and slightly more than % of 
a year’s growth in paragraph meaning. This careful teaching of the 
meaning of the more difficult words stands out as one good way to 
make an assignment. Another method almost as effective is the 
development: of a background for understanding the material to be 
studied. This background of understanding is of undoubted assistance 
in comprehending the meaning of the material studied. 
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Since these assignments were tried out upon Grades V, VII, and IX, 
opportunity was offered to study the differential effect of the assign- 
ment upon each grade. 


TasLe VII.—ComparaTIvE EFFECTIVENESS OF THE Five Metuops or MAKING 
THE ASSIGNMENT ON Firtu, SEVENTH, AND Ninta Grape Pupiis 





| Grade V | Grade VII Grade IX 


Word knowledge.................... | C | C C 








Sentence meaning................... | E | E B 


Paragraph meaning..................| D | E | B 





Teaching the meaning of the difficult word (Situation C) caused the 
greatest gain in word knowledge for each of the three grades tested. 
Making pupils aware of the general meaning of the materials (Situation 
E) caused the greatest gain in sentence meaning on the part of fifth and 
seventh grade pupils while with the ninth grade it was the assignment 
of pertinent questions (Situation B). In developing paragraph mean- 
ing, making the pupils aware of the personal value of materials appears 
to be best in the fifth grade; while making pupils aware of the general 
background of the material studied is most successful in the seventh 
grade. 

While the gains in the experimental situations over the control 
situations are not statistically significant, except in two cases of word 
knowledge (Situations A and C), yet our dav show strong indications 
of the influence of the assignment. ‘The consistency of the gain, the 
fact that each type caused a noticeable gain on the part of one or two 
of the grades examined, the brief time for acquainting the pupils with 
the specific procedure used in the assignment, and the large per cent of a 
year’s gain in many cases, are evidences of the importance of the 
assignment. 
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FACTUAL MEMORY OF SECONDARY SCHOOL PUPILS 
FOR A SHORT ARTICLE WHICH THEY READ A 
SINGLE TIME 


ALFRED G. DIETZE AND GEORGE E. JONES 
University of Pittsburgh 
(Concluded from November Issue) 


FORGETTING 


Comparison of Group Averages.—Table VII repeats the averages 
presented in Tables I to V' in such a manner as to reveal the influence 
of lapse of time to bring about forgetting of an article read a single 
time. It is clear from these figures that with passage of time pupils 
remember fewer and fewer of the facts contained in the articles. 
Grade by grade and article by article eighty-four comparisons may be 
made between successive intervals. With six exceptions, only one of 
which is large enough to be significant, the averages of any given 
interval are smaller than those of the intervals preceding. This is 
noteworthy in view of the fact that the figures are based on groups of 
varying abilities. 

Table VIII gives the A-score averages for the combined groups.? 
Average scores decrease consistently with time in the figures of this 
table. The conclusion seems warranted that, within the limits 
of the present study, the greater the interval between the reading of an 
article and the subsequent attempt to recall its content the greater is 
the forgetting for the facts contained. 

Results of Parallel Groups.—The above findings were checked for 
accuracy by using the method of equivalent groups. One hundred 
cases were selected from each of the six experimental groups on the 
basis of A-scores in immediate memory in such a manner that the six 
resulting groups were exactly equivalent and that the A-scores of 
each were normally distributed. Since 56.7 happened to be the 
A-score average of the combined grades, we selected 56.0 as the mean 
and 8.6 as the standard deviation of the new parallel groups and then 
computed the theoretical frequencies for six class intervals of four 





1 See first half of this article in previous issue. 
2It should be recalled that the A-score average represents the work of all 
groups taken together in terms of the material Arkwright. 
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points each above and below the mean in a normal curve with mean 
and standard deviation as above and area equal to one hundred. The 


Taste VII.—AveraGe Memory AFTER DIFFERENT INTERVALS! 

















Grades 
Recall interval 

VII | VIII; IX x XI | XII | All 

Germans 
RRs 4 53.8 | 57.9 | 67.3 | 66.6 | 73.0 | 75.8 | 64.8 
ees cin ss hats 8 kira 49.0 | 55.1 | 54.2 | 58.8 | 59.3 | 71.8 | 56.7 
Fourteen days.............. 37.9 | 34.3 | 38.8 | 44.2 | 46.7 | 51.3 | 42.0 
NS cao) pies acs 4s 32.9 | 41.3 | 32.1 | 32.1 | 40.1 | 39.3 | 36.4 
One hundred days........... 29.4 | 26.5 | 31.4 | 35.9 | 33.3 | 32.4 

Radium 
ETL ES POLES 67.3 | 66.6 | 72.4 | 76.6 | 77.0 | 84.3 | 73.4 
a a ote | 47.7 | 55.0 | 56.1 | 56.3 | 64.9 | 72.9 | 58.7 
Fourteen days.............. | 40.7 | 45.2 | 44.9 | 40.8 | 50.6 | 40.1 | 43.7 
ee a oa Sas o's 64.0 37.2 | 37.7 | 37.1 | 42.2 | 42.1 | 37.9 | 38.3 
One hundred days...........| 27.3 | 31.9 | 38.6 | 36.6 | 35.1 | 39.1 | 33.8 

Arkwright 
Ea ARR 43.9 | 50.0 | 50.3 | 57.4 | 62.7 | 64.8 | 54.6 
e's ks apa gare 33.5 | 43.4 | 42.4 | 49.7 | 55.2 | 63.3 | 46.4 
Fourteen days.............. 87.5 | 28.8 | 32.9 | 34.4 | 32.3 | 32.0 | 33.1 
- a a a 25.9 | 27.6 | 25.4 | 37.8 | 31.5 | 31.3 | 30.3 
One hundred days........... 20.5 | 22.0 | 21.9 | 28.7 | 23.0 | 22.6 | 23.4 























1 Italicized figures show scores that are higher than preceding interval. 


Taste VIII.—Averaces MEmorY AFTER DIFFERENT INTERVALS—A-SCORE 
































Grades 
Recall interval 

VII | VIII; IX x XI | XII | All 
RS a's ad eve'e bo 50.0 | 51.3 | 55.5 | 59.0 | 62.3 | 65.2 | 56.7 
| ERT ee ee 40.0 | 45.3 | 46.0 | 47.9 | 53.2 | 61.1 | 48.0 
Fourteen days.............. 36.5 | 33.8 | 35.9 | 36.1 | 39.0 | 36.8 | 36.3 
i ns cad iames 25.5 | 33.2 | 29.5 | 35.7 | 34.9 | 33.8 | 32.4 
One hundred days........... | 25.5 | 27.9 | 28.5 | 30.5 | 29.6 | 31.2 | 30.0 





hundred cases of each group were then taken to fit these frequencies. 
An attempt was made to keep the number from each grade approxi- 
mately the same. 








l- 
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The reader should note that this method makes possible the com- 
parison of different intervals on the basis of a given material even 
though a different group was used for each interval ;—e.g., in the case 
of Radium we may compare the results of Group I or II in immediate 
memory with Group V in memory after one day, Group VI in memory 
after fourteen days, Group IV in memory after thirty days, and Group 
III in memory after one hundred days. The six groups are almost 
exactly alike in terms of A-scores in immediate memory. 

The results for each material are presented in Table IX. The 
reader should note especially the close correspondence between the 
A-score column of the present table with the last column of Table 
VIII. Such close correspondence between two methods of analysis 
is an indication of the validity of the methods. 


TaBLeE I[X.—AveRAGE MeEMorY OF PARALLEL GROUPS AFTER DIFFERENT 














INTERVALS! 
Interval | Radium | Germans | Arkwright | A-score 
| 
Ns. cha kaa ean bd kak a8 .i @.7 56.0 | 56.0 
TTD ae AE wk 49.4 a3 | @.8 
ss. onc casec-e een 45.0 37.7 33.0 35.6 
I 6 i ciesu'g io 6 wd cori nd 41.2 36.9 26.2 32.4 
One hundred days.............. 35.4 32.7 22.4 28.7 











1 Each average is based on one hundred cases, the A-scores on three hundred. 


Rate of Forgetting —A curve of forgetting may be plotted either 
from the data of the last column in Table VIII or from those of the 
last column in Table IX. Such a curve would be representative of the 
forgetting of pupils in Grades VII to XII for the article “Arkwright” 
after a single reading. The data of Table VIII are plotted in the 
forgetting curve of Fig.3. The curve is begun at sixty-three per cent, 
since this is approximately the amount of the test which pupils in these 
grades can answer when they are permitted to look up the answers in 
the text,—.e., sixty-three per cent of the article “gets across” in a 
single reading (see supra. paragraph entitled, “Comprehension Dif- 
ficulty and Carelessness”). To enable the reader also to determine 
how much of what actually “gets across’’ in a single reading is remem- 
bered at each of the intervals, a second scale is provided to the right of 
the graph which begins the curve at one hundred per cent. The 
asymptote drawn in represents the estimated point of complete for- 
getting for the material, which is taken at six per cent since it was 
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found that pupils could answer this amount of the test without pre- 
viously having read the article (see supra., paragraph entitled ‘‘Previ- 
ous Knowledge’’). 

Summarizing and interpreting the data presented, we find that 
pupils in the grades studied are able to answer an average of sixty-three 
per cent of the questions on Arkwright when they are permitted to 
look up the answers in the article; that immediately after the reading 
they remember an average of 56.7 per cent; after one day an average of 
48.0 per cent; after fourteen days an average of 36.3 per cent; after 
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* Fie. 3.—Average forgetting curve for Arkwright based on A-Scores of all groups par- 
ticipating in the experiments. 


thirty days an average of 32.4 per cent; and after one hundred days an 
average of 30.0 per cent. ‘The immediate memory score represents an 
average loss of 43.3 per cent of the facts presented in the article read; 
of this, the difference between one hundred per cent and sixty-three, 
or thirty-seven per cent, is due to failure to comprehend and to careless 
working habits, and the remainder, 6.3 per cent, to forgetting during 
the interval of reading the article and taking the test. If we com- 
pute the per cent remembered of that which actually “gets across,” 
we obtain scores as follows: Immediate memory 90.0 per cent, memory 
after one day 76.2 per cent, memory after fourteen days 59.3 per cent, 
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memory after thirty days 51.7 per cent, and memory after one hundred 
days 47.6 per cent. 

We find here the same principle discovered by Ebbinghaus,—7z., 
that in forgetting there is a rapid initial loss and a more gradual 
subsequent loss. We find also the principle corroborated that mean- 
ingful material is forgotten less quickly than nonsense syllables; and we 
may add that facts learned in a single reading of an article of the diffi- 
culty of Arkwright are, in the longer intervals at least, forgotten less 
rapidly than in meaningful material in the form of poetry learned to the 
degree of one correct and unhesitating recital. In Table X we present 
data taken from three classic studies of forgetting! and corapare them 
with our results. A study of the table shows that the amounts for- 
gotten of what is actually apprehended of Arkwright are less, in general, 
than for the other materials, especially after intervals of more than one 
day. From the position of the asymptote in Fig. 3, it is also apparent 
that the point of complete forgetting is still far off even after one 
hundred days, and, from the shape of the curve, that it would take a 
long time to reach this point, if it were ever reached. 


TaBLE X.—MeEmorY AFTER DIFFERENT INTERVALS 
Comparison of Results of Ebbinghaus, Radossawljewitsch and Finkenbinder with 











Present Study 
Nonsense syllables Poetry | Factual 
Inte 
—— Epbing- | P40" | Finken-| B08 | 
haus | @7Y& | binder | US ane 
witsch witsch 
Immediate (20 minutes)........ 58 89 232 96 90 
NS hed avn cei snp akc nds 34 68 58 79 76 
ns. oo ab vie 6 bs 22! 41 473 30 59 
al... Laas saa’ 21 20 44? 24 52 
One hundred days.............. ? 41 38? ? 48 




















1 By interpolation in author’s tables. 
* By entry in Finkenbinders tables of 10 Log. 





1 Ebbinghaus, H.: Ueber das Gedaechinis. Leipzig, 1885: translated by H. A. 
Ruger and E. Bussenius, Memory. Bureau of Publications, Teachers College, 
Columbia University, New York, 1913. 

Radossawljewitsch, P. R.: Das Fortschreiten des Vergessens mit der Zeit. 
Druck der Dieterich’schen Universitaet, Buchdruckerei, Goettingen, 1905. 

Finkenbinder, E. O.: The Curve of Forgetting. American Journal of Psy- 
chology, Vol. XXIV, 1913, pp. 8-32. 
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The conclusion is reached on the basis of the above analysis, that 
(a) forgetting of factual material read a single time follows the same 
general law of rapid initial loss and more gradual subsequent loss as was 
found by other investigators for rote memory; (b) forgetting of factual 
material proceeds more slowly than forgetting of nonsense syllables; 
(c) forgetting of factual material proceeds more slowly than forgetting 
of meaningful material learned by rote; and (d) it takes much longer to 
approximate a point of complete forgetting for factual material than it 
does either for nonsense syllables or meaningful materia! learned by 
rote. It should be remembered that the above comparisons are 
between memory for facts apprehended in a single reading of an article 
as measured by a recognition test and verbatim memory for materials 
learned by rote to the point of one perfect and unhesitating 
reproduction. 


CORRELATIONS BETWEEN MEMORY INTERVALS 


Introduction.—The problem of forgetting will be still further 
clarified by an analysis of the relationship existing between the scores 
of the same individuals after different intervals,—+.e., of the question, 
“Do pupils rank in the same order, in memory after different time 
intervals?’’ This can be done most expeditiously by the correlation 
method. 

Several workers have investigated the correlation between immedi- 
ate and delayed memory. Among these, Thorndike found a correla- 
tion of .9 between immediate recall for twelve unconnected words and 
delayed recall after twenty-four hours.! Gates found correlations 
of .73 to .89 between immediate and delayed recall of nonsense syl- 
lables and of biographies. His subjects were children in the elemen- 
tary school.2 Bassett reports average correlations of .838 between 
immediate recall of history courses and recall after four months, .797 
between immediate recall and recall after eight months, .839 between 
immediate recall and recall after twelve months, .840 between immedi- 
ate recall and recall after sixteen months, .935 between recall after 
four and eight months, .915 between recall after eight and twelve 
months, and .961 between recall after twelve and sixteen months. 


1 Thorndike, E. L.: The Relation between Memory for Words and Memory for 
Numbers, and the Relation between Memory over Short and Memory over Long 
Intervals. American Journal of Psychology, Vol. XXI, 1901, pp. 487-488. 





2 Gates, A. I.: Correlations of Immediate and Delayed Recall. Journal of | 


Experimental Psychology, Vol. IX, 1918, pp. 489-496. 
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She concludes that time and forgetting do not greatly reduce the 

relative standing of pupils in retention of history learned in school.! 
The correlations obtained by the authors cited are, in general, 

fairly high. Indeed, Thorndike makes the statement, “The relation 


TaBLeE XI.—CoORRELATIONS BETWEEN FactrvuaL MEMORY AFTER VARIOUS 














INTERVALS 
r corrected 
V, Variables r PE for N 
| attenuation 

Immediate Radium; one day Germans......... .75 .017 .85 293 
Immediate Germans; one day Arkwright.......| .78 , .016 . 83 284 
Immediate Arkwright; one day Radium........| .73 | .019 .80 286 

Average: Immediate; one day............... we Fink: 83 
Immediate Radium; fourteen days Germans....| .61 | .026 .66 273 
Immediate Germans; fourteen days Arkwright..| .50 | .030 .54 289 
Immediate Arkwright; fourteen days Radium...| .53 | .029 .58 282 

Average: Immediate; fourteen days..........| .55 | .... .59 
Immediate Radium; thirty days Arkwright...... .48 | .032 .53 268 
Immediate Germans; thirty days Radium.......| .50 | .030 54 288 
Immediate Arkwright; 30 days Germans........| .58 | .030 .62 225 

Average: Immediate; thirty days............ ES ane . 56 
Immediate Radium; one hundred days Arkwright) .36 | .039 .39 223 
Immediate Germans; one hundred days Radium| .48 | .032 .52 261 
Immediate Arkwright; one hundred days Ger- 

DCAVES STEEL hak ceed Ge ee cheese es ba:0 .50 | .036 .54 204 

Average: Immediate; one hundred days... ... : | eee .48 
Fourteen days Radium; thirty days Germans..., .58 | .032 sf 192 
Fourteen days Germans; thirty days Arkwright..| .58 | .030 ate | 223 
Fourteen days Arkwright; thirty days Radium... | .50 | .033 8 | 241 

Average: fourteen days; thirty days..........| .55 | .... , 
One day Radium; one hundred days Germans...) .33 | .056 3 115 
One day Germans; one hundred days Arkwright} .33 | .062 eb2 | 93 
One day Arkwright; one hundred days Radium..| .38 | .053 vile 119 

Average: one day; one hundred days......... .35 

















between retention of the effects of an experience for one or two minutes 
and their retention for one or two days thus seems to be one of the 
closest yet measured in human nature.’”’ However, we must remem- 





1 Bassett, S. J.: ‘‘Retention of History in the Sixth, Seventh and Eighth 
Grades with Special Reference to the Factors That Influence Retention.’’ Johns 
Hopkins Studies in Education No. 12. The Johns Hopkins Press, Baltimore, 
1928, pp. 22-24. 
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ber that a correlation of as much as .90 reduces the standard error of 
estimate to only a little less than the standard deviation. 

Result of Present Study.—The correlations found between the 
various memory intervals of the present study are shown in Table 
XI. Each correlation is based upon the largest number of cases 
available for the purpose. Since different materials were used at the 
intervals compared, the coefficients are attenuated. Corrections have 
been made for attentuation, and these are presented in a separate 
column. Without doubt the corrected coefficients are much nearer 
the truth than the raw coefficients; however, even they are far from 
perfect. 

The correlations here reported indicate a fair degree of positive 
correlation between the various intervals. A study of the table 
reveals an interesting fact,—wz., that the correlation coefficients 
become smaller the greater the interval between the memories measures. 
The averages of the correlations are in the following order: 














, oo Be : Average 
Variables Time intervening Pe ~ 
Immediate; one day............... one day | .75 
Immediate; fourteen days.......... fourteen days .55 
Fourteen days; thirty days......... sixteen days .55 
Immediate; thirty days............. thirty days .52 
Immediate; one hundred days....... one hundred days 45 
One day; one hundred days......... ninety-nine days .35 





It seems, from these facts, that the relationship existing between 
memory at different intervals is influenced the more by uncorrelated 
factors as time goes on. This problem deserves further investigation. 
The writers offer the hypothesis that, among other factors, guessing 
plays a larger and larger réle with the increase of time interval between 
reading and testing. This conjecture, indeed, is supported by the fact 
revealed by an examination of the test papers, that the subjects make 
fewer attempts the longer the intervals, indicating that they are more 
certain of their answers in the shorter periods than in the longer ones. 


SuMMARY OF RESULTS 


1. Immediate and delayed factual memory for an article read a 
single time varies with the material used, the more familiar articles, 
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Radium and Germans, being remembered better than the less familiar 
Arkwright. 

2. Negative skewed distributions in immediate and one day delayed 
recall for Radium and Germans indicate a piling up of scores in the 
range above the mean; Arkwright gives normal distributions at these 
intervals. 

3. Wide individual differences, extending over about three-fourths 
of the scoring range, are present in the abilities of subjects to remember 
what they read a single time. 

4. Memory for periods longer than one day result in slightly 
positively skewed distributions. 

5. Immediate memory for an article read a single time increases 
with grade from fifty per cent in the seventh to sixty-five per cent in the 
twelfth for the material ‘Arkwright,’ and similarly for other materials. 

6. One day delayed memory for Arkwright likewise increases with 
grade from forty per cent in the seventh to sixty-one per cent in the 
twelfth. 

7. No marked relationship exists between grade and factual 
memory over fourteen days and longer. 

8. Factual material is forgotten similarly to nonsense syllables 
and meaningful materials learned by rote,—viz., rapidly at first, then 
more and more slowly as the interval between learning and the measure- 
ment of memory increases. 

9. When memory is based on the amount of the article actually 
apprehended by the pupils, the initial forgetting for factual material 
is less abrupt than for nonsense syllables. 

10. Factual material is forgotten less rapidly than either nonsense 
syllables or meaningful material learned by rote. 

11. The point of complete forgetting is not nearly approximated 
even after one hundred days. 

12. An appreciable amount (six per cent for Arkwright) is for- 
gotten during the time of reading and writing the test. 

13. Positive correlation exists between immediate memory and 
delayed memory, also between delayed memory at different intervals. 

14, The degree of correlation existing between memory at different 
intervals vary inversely with the length of the interval. They 
range from an average of .75 between immediate and one day 
delayed memory to .35 between one day and one hundred day delayed 
memory. 
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CoMMENTS 


1. Since the type of material which pupils read and study markedly 
influences the amount which they apprehend and remember, methods 
should be devised for scientifically matching the difficulty of text- 
books to the ability level of the pupiis for whom they are intended. 

2. More research is needed to determine the factors inherent in 
material read and in the mental make-up of pupils which determine the 
efficiency of memory. 

3. It is obvious that a single reading of lesson assignments is an 
inefficient method of study. Studies are needed to discover more 
effective study techniques. The influence of repeated readings, 
summarizing, reading to find answers to questions and to solve prob- 
lems, are suggested problems. A veritable mine of similar problems 
are suggested throughout the pages of G. A. Yoakam’s recent book, 
Reading and Study. 

4. Other interesting fields of experimentation are suggested by 
statements nine, ten, eleven and fourteen of the foregoing summary. 





1The Macmillan Co., New York, 1928. 





EFFECT OF ORDER OF PRESENTATION ON THE 
RECALL OF PICTURES 


DANIEL D. DROBA 


Ohio State University 


Results described in this report were obtained in connection with 
an experiment on the effect of printed information on memory for 
pictures... One hundred twenty students attending the University 
of Chicago were used for subjects. For memory material twenty 


uncolored postcard pictures of paintings in the Chicago Art Institute 
were used throughout as follows: 


1. A Woman in Gray, by William Orpen, English, 1878— (14) 

2. The Fountain, by John S. Sargent, American, 1856-1925. (16) 

3. Joan of Arc at the Court of Chinon, by Maurice Boutet De Monvel, 
French, 1850-1913. (15) 
Arab Scouts, by Adolph Schreyer, German, 1828-1899. (20) 
The Music Lesson, by Gerard Terberg, Dutch, 1617-1640. (19) 
The Young Duchess, by John Faed, Scotch, 1820-1902. (10) 
Marches in the North of Holland, by Eugene Jettel, Austrian. 1835- 
1901. (5) 

8. The Potato Harvest, by Ludwig Knaus, German, 1829-1910. (11.5) 

9. Democritus, The Laughing Philosopher, by Jusepe Ribera, 1588-1656. (17) 

10. Portrait of Helena Dubois, by Anthony Van Dyck, Flemish, 1599-1641. 
(18) 

11. Rembrandt’s Father, by Rembrandt Harmensz Van Rijn, Dutch, 1606- 
1669. (2.5) 


12. The Wrestler’s Challenge, by Mihaly Munkacsy, Hungarian, 1846-1900. 
(11.5) 


13. Night in the Garden of Gethsemane, by Louis Cranach, German, 1472- 
1553. (7) 

14. A Wrangle Over Cards, by Jean David Col, Belgian, 1822-1900. (1) 

15. The Assumption of the Virgin, by Domenikos Theotokopoulos, Spanish, 
1547-1614. (4) 

16. The Repentance of Simon Peter, by Jean Charles Cazin, French, 1841- 
1901. (6) 

17. Pilots, Puerto De Los Pasajes, Spain, by Frank W. Brangwyn, English, 
1867-— (13) 

18. He That Is Without Sin Among You, by Benjamin West, American, 
1738-1820. (9) 

19. The Water Mill, by Meindert Hobbema, Dutch, 1628-1709. (8) 

20. In Holland Waters, by Paul Jean Charles Clays, Belgian, 1819-1900. (2.5) 


NP 





1D. D. Droba: Effect of Printed Information on Memory for Pictures. 
Museum News, Sept., 1929; Psych. Abstracts, No. 94, 1930. 
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Each picture was fastened on a white card and the information 
pertaining to the picture was typed either at the bottom or to the 
right of the picture. Six types of information were used ranging from 
no information to an information including the title of the picture, 
the last name of the painter, and a sentence about the painter. 

The pictures were shown individually to every subject, the exposure 
time being fifteen seconds for every picture. Immediately after all the 
pictures were presented the subject was asked to describe on a sepa- 
rate card all the pictures he was able to recall. Twenty subjects were 
used for each of the six conditions of information. 

For studying the relation between the order of presentation and 
frequency of recall two methods were used. The first was the rank 
correlation method. Hence, we ascertained the number of times a 
picture was recalled by the one hundred twenty subjects. The 
pictures were then ranked on the basis of the frequency of recall 
The most frequently recalled picture was put first, the next most 
frequently remembered picture was put the second, and so on (rank 
numbers of recall are given in the parenthesis after each picture in 
the list). The rank correlation between the frequency of recall and 
the order of presentation was found to be —.61. This negative corre- 
lation tends to show that the earlier a picture is presented the less 
likely it will be recalled. 

The second method used was the graphic method. The twenty 
pictures were divided into two groups: The ten pictures presented 
first and the ten pictures presented second. The averages of the num- 
ber of pictures recalled under each of the six conditions of information 
was calculated. These averages, together with the total averages for 
all the one hundred twenty subjects, were tabulated in Table 1. 

















TaBLeE I 
Pictures I II Ill | IV V VI | Total 
ae ai ceia skies ann 12.6 | 13.3 | 12.4 | 11.3 | 10.0 | 11.6 | 11.8 
b. Second ten..............4 15.3 | 17.1 | 14.9 | 14.0 | 12.6 | 15.1 | 14.8 
| PT eee err ere 20 20 20 20 20 20 {120 


























Under each condition of information, designated by Roman 
numbers from I to VI, a higher average was found for the second half 
of the series of pictures. The difference between the total averages 
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for all the one hundred twenty subjects is 3.0 which is a confirmation 
of the above finding that pictures presented in the second half of the 
series were recalled more frequently, on the average, then pictures 
presented in the first half of the series. 

Figure 1 graphically represents these averages plotted against the 
amount of information. Curve a represents the first ten pictures and 


i A FIGURE 1 





curve 6 the second ten pictures. The difference between the two 
curves is readily seen: b is much higher in all conditions than curve a. 
This means that pictures located in the second half of the series were 


consistently recalled more times than those in the first half of the 
series. 
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The two halves of the pictures were then again divided into two 
equal subgroups so that we obtained four groups of five pictures 
each. Similarly, the averages of recall were calculated for each group 
under each condition, and tabulated in Table II. 


i RY FIGURE 2 
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The total averages for all the one hundred twenty subjects are 
also included in the table. According to the total averages (the last 
column) pictures in the third quarter were recalled the most frequently. 
Pictures located in the last quarter come next, the second group the 
next, and the first group is the last of all. 
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Data shown in Table II were plotted in Fig. 2. The highest 
curve is curve c representing the third group of five pictures which 
indicates that the best position is the third quarter in all except the 
first and the fifth conditions. In the latter condition it is equal 
with the last quarter. Curve d representing the last quarter drops 
down especially in the medium conditions. Curve b representing 
the second quarter is different from the first two especially under 
the end conditions. 











Taste II 

| l 
Pictures | I II | III IV | V | VI | Total 
Gees... .......- oe | 11.9| 13.0/ 11.9] 9.6] 8.0] 10.9| 10.8 
b. Second five..............| 18.4 | 13.6 | 13.0 | 13.0 | 12.0 | 12.4 | 12.9 
c. Third five. . ........| 15.2 | 17.2 | 16.0 | 15.0 | 12.6 | 15.6 | 15.2 
d. Fourth five.............. 15.4 | 17.0 | 13.6 | 13.0 | 12.6 | 14.6 | 14.3 

iene ta Mia 20 | 20 | 20 | 20 | 20 | 20 /|120 























Pictures located in the first quarter and symbolized by curve a were 
decidedly less frequently recalled than pictures placed in the other 
three quarters. This fact is clearly evident from the figure. 

An additional fact may be noted from both figures which is not 
strictly related to our problem. The relation of the amount of 
information and frequency of immediate recall is consistent for 
all the groups of pictures: the halves and the quarters. All curves 
first rise to the second condition, then they drop down to the 
fifth condition, and again rise for the sixth condition. The meaning 
of this relation was substantiated in a separate article.' However, 
the consistency of the relation is being mentioned here for the 
first time. 

The following conclusions may be drawn from this experiment: 

1. The rank correlation between the order of presentation and the 
frequency of immediate recall seems to indicate. that priority of 
presentation is not conducive to such recall. 

2. In all the six conditions of information the position of a picture 
in the second half of the series is more favorable to immediate recall 
than in the first half. 





1 Ibid. 
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3. Pictures located in the third quarter of the series were recalled 
the most frequently of all. 

4. Position in the first quarter is decidedly less favorable to immedi- 
ate recall than positions in the other three quarters. 

5. Effect of information on memory for pictures, described in a 
previous article, was found to be consistent for all the above mentioned 
positions. 


THE INFLUENCE OF AN AUDIENCE UPON RECALL 


CLARA BURRI 


Graduate Student, University of Chicago 
THE PROBLEM 


Introspective reports from many observers indicate that the pres- 
ence of an audience frequently has a detrimental effect upon perform- 
ance. Many individuals relate that the presence of others causes 
them many difficulties of behavior especially in speech and mnemonic 
responses. 

Sometimes we find that a student comes to school, seemingly 
unprepared, yet he maintaines that: ‘‘He did learn his lesson, but 
that he just could not remember when others were watching.” Every 
teacher is ‘amiliar with this or similar statements and he frequently 
considers it as a mere excuse for not having studied. This investiga- 
tion is an attempt to study experimentally some of the fundamental 
phases of the above problem. Our task is to discover whether an 
individual who has learned certain memory material, and is then 
required to recall that material before a group of people, will as a 
consequence of the presence of that audience, be able to recall as 
readily as when he is alone. 

Few people have considered the influence of a group on an indi- 
vidual’s efficiency to recall when learning did not take place in that 
group. Elkine! is the only author who reports any data concerning 
group influence on recall or reproduction. He reports that memory 
is better in the group. Forty children were given one presentation of 
a series of one-syllable words and were then called upon for a written 
reproduction immediately after the presentation, and another one 
day later. First, the individuals went through the above stated pro- 
cedure alone, and later as members of a group. Elkine explains his 
results in terms of “rivalry and the influence of suggestibility due to 
contact and mere presence of other people.” Elkine’s study differs 
from the one reported in this paper in one important aspect. He 
studied the influence of a group on learning and memory when the 
subject was a member of the group. The results obtained may there- 
fore be explained as being due to rivalry. In this study particular 


1 Elkine, E.: Influence de group sur la memoire. J. de Psychol., 1927, pp: 
827-830. 
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care was taken to eliminate, as far as possible, the factor of competition, 
and hence the subjects were tested when recalling before people who 
were not engaged in any comparable activity. 


METHOD AND PROCEDURE 


Subjects.—Sixty students, nineteen men and forty-one women, 
from the classes in introductory psychology, and from other depart- 
ments of the University of Chicago, participated in the experiment. 

Conditions.—The subjects were divided at random into three 
groups of twenty each. Group one acted as control, learning and 
recalling under the normal condition, with only the experimenter 
present. Group two learned under the same condition as did group 
one, but they recalled and relearned before an audience consisting of 
four persons who sat in the experimental room without paying any 
attention to them. The individuals of group three also learned alone, 
and then recalled and relearned before an audience. This audience 
however attentively watched and kept track of their responses. 

Audiences.—The audiences consisted of two men and two women, 
all graduate students or majors in psychology. They were selected 
with a view to their prestige in the eyes of the subjects, and as far as 
possible consisted of people who were not acquainted with the sub- 
jects. No attempt was made to keep the same individuals for spec- 
tators throughout the experiment. As has already been suggested 
two kinds of audiences were employed: One in which the members 
paid no attention to the subject, but were merely present in the experi- 
mental room: Another, an attentive audience, the members of which 
listened to the subject and checked his responses on a slip of paper 
which contained the lists of words in their correct order. 

Material.—The material to be learned consisted of a list of fifteen 
pairs of words which had been selected at random from a standard 
dictionary. In constructing the list care was taken that no letter 
should occur twice in a given pair. The words were exposed by means 
of a hand-operated inemory drum, a standard instrument in the 
laboratory of the University of Chicago. Other materials used were 
cards three to five inches with the response words copied on incorre- 
sponding order with the stimulus words of the test series, for recording 
the responses. A stopwatch served for timing the exposure of the 
paired associates. 

Procedure.—The paired associate list was presented once, allowing 

seconds for each exposure, after which the subject was tested for 
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recall. In the test series each stimulus word was presented alone and 
in a different order from that in which it originally appeared, and the 
subject had to give the proper response word within the given time of 
three seconds. This entire procedure was continued until the subject 
was able to respond correctly to every stimulus word in the list. The 
criterion of learning was one errorless recall. A subject learned 
the list of paired associates and then recalled and relearned them 
twenty-four hours later either before the experimenter alone, or in 
presence of an audience, attentive or inattentive, depending upon the 
particular condition desired. 

Instructions.—At the beginning of the experiment each subject was 
required to read a carefully prepared and typewritten instruction 
sheet, after which he was quizzed briefly to make certain that he 
understood the general nature of his task. At the same time he was 
also told under what condition he was expected to recall, and relearn 
on the following day. Thus every subject knew before he started to 
learn, whether an audience, and if so, what kind of an audience, was 
to be present for recall. 

The only subjects given other instructions on the recall day were 
those who were to face an attentive audience. To them, the request 
was made to ‘‘respond distinctly and loudly, so that every person 
in the room can understand.” This extra order was given to enhance 
the effectiveness of the attentive audience and thus to further increase 
the difference between the non-attentive and attentive audience. 

Treatment of Data.—The amount of retention is stated in terms of 
(1) number of words recalled; (2) number of trials required to relearn; 
and (3) the saving score, which was obtained by the following formula: 


“ee Trials to relearn 
Trials to learn 





The difference in efficiency of recall between the several conditions is 
determined statistically and the results are stated in terms of the 
numerical differences, the averages, the probable errors of the means, 
the difference between the means, and the probable error of their 
differences. 

RESULTS 


Table I gives the total and mean number of words recalled, trials 
required to relearn and the saving score for the different conditions. 
If we analyze the data it appears that the results in terms of recall are 
most clear cut, while those in terms of relearning and saving, although 





#4 


Mt | 
- 
i, 
| 
j 








+ tale er ie 
ea 9 ey ye 
ee 





686 The Journal of Educational Psychology 


less large, still show a definite tendency. The efficiency of retention 
as measured in terms of recall, relearning, and saving, is greatest for 
no audience. In the case of an audience, and an attentive audience, 
the numerical scores for retention are considerably smaller than those 
for no audience, while on the other hand they are about equal for the 
two kinds of audiences. The statistical significance of the findings is 
shown in the later tables. These results throw light upon a funda- 
mental principle of reproduction. They indicate that any stimulus- 


TaBLe I.—Tue Erricrency or RETENTION IN THE DIFFERENT CONDITIONS 








Recall; | Relearning; eae g; 
Condition syllables trials to . a tage 
recalled relearn shone eae ae 
No audience. 
DE SIO as Src c  ooebk See cue oan owas 269 18 1561 
eI oe OF em ee Ba aS a se 13.45 .90 78.05 
Audience. 
RRR a VS ae ae a See 237 33 1387 
is aos sks 44400 He een ewhe 600606 11.85 1.65 69.35 
Attentive audience. 
REE Ee aera Oe eee are Peep ne eer 234 32 1417 
EE ne | Ee Py om 11.70 1.60 70.85 














response situation must be considered as a functional whole consisting 
of a constellation of numerous organismic and environmental factors. 
Any change in the situation will affect the synthesis of the whole. If 
a behavior pattern is once formed it may be reinstated best in a similar 
situation to that in which it was originally acquired. In our problem 
not only the total organization present at the time of learning, but the 
constellation of organismi¢c and environmental factors at the time of 
recall were equally important in facilitating or inhibiting reproduction. 

Table II shows the mean number of words recalled, of trials 
required to relearn, and of the saving per list for each of the two con- 
ditions, no audience and audience; the probable errors of the two 
means from the three criteria and the probable errors of their differ- 
ences. There is a difference between the two conditions in all of the 
three criteria. The one in terms of recall is almost four times its 
probable error. The differences in the case of the relearning and 
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saving scores is not so great, yet they indicate a consistent tendency. 
This is especially remarkable since these scores are not direct measures 
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of recall. 
TasLe II.—Tue DirFrerRENCE BETWEEN No AUDIENCE AND INATTENTIVE 
AUDIENCE 
; Saving; 
Recall; | Relearning; : 
Condition syllables | trials to pe 
recalled relearn enti aed oo 
No audierce. 
es bition lee Cin wuldak baeeta 13.45 .90 78.05 
SEE CRS er nS ee ee Oe .28 .12 4.14 
Audience 
i ates hs iii lait ad EN ta 11.85 1.65 69.35 
kg le a il a a .33 .21 4.67 
Difference between the two means........ 1.60 .75 8.70 
| TERS BOER BARAT SES Lae 43 .24 6.24 














In Table III we find the mean number of the words recalled, of 
trials required to relearn, and of the saving per list for each of the two 
conditions, no audience and attentive audience, the probable errors of 
the two means from the three criteria, and the probable error of their 








differences. 
TaBLe III.—Tuze DirreRENCE BETWEEN No AUDIENCE AND ATTENTIVE 
AUDIENCE 
Recall; | Relearning; aie 
Condition syllables trials to - f rs waar 
recalled relearn Chsiuieion a 
No audience. 
a i 13.45 .90 78.05 
a i a 8 ee ee .28 .12 4.14 
Attentive audience. 
Lo ie Me okt 11.70 1.60 70.85 
I Ae iat a te ee wale alee & .17 .16 4.23 
Difference between the two means........ 1.75 .70 7.20 
ERTIES aa ae ee pee .32 .20 5.91 














As before we find that the subjects with audience give poorer 
The difference between no audience and attentive audience 


results. 
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is even larger than is the difference between no audience and inatten- 
tive audience. Again we get the largest divergency for recall with a 
disparity between the two averages of 1.75 and a probable error of the 
difference of .32, which is less than one-fifth of the actual difference. 
This indicates a high degree of significance. 

The results become more striking when we consider that the differ- 
ences exist in spite of the fact that the subjects took, on an average, 
longer to learn the paired associates when they were anticipating an 
audience, than when expecting to recall only before the experimenter. 
We generally find that a longer learning period is followed by better 
recall. One can scarcely avoid the inference that this lengthened learn- 
ing period is an outcome of the expectancy of an audience. The dis- 
cussion of this topic, however, will have to be left until later. 


TasBLeE IV.—Tue DIFFERENCE BETWEEN INATTENTIVE AUDIENCE AND 
ATTENTIVE AUDIENCE 

















Recall; | Relearning;| 5*¥28: 
Condition syllables | trials to | Percentage 
recalled relearn of learning 
time saved 
Audience. 
RE Os. bas choos s 11.85 1.65 69.35 
Bebe dees ae. oki oo ee .33 .21 4.67 
Attentive audience. 
ea i Gm a hale a a ae 11.70 1.60 70.85 
RAE SEES Cae pn eT ae 17 .16 4.23 
Difference between the two means........ .15 .05 1.60 
payee nS Paice Ras a re .37 . 26 6.30 





Table IV gives the mean number of words recalled, of trials required 
to relearn and of the saving per list for each of the two conditions, 
inattentive and attentive audience; the probable errors of the two 
means from the three criteria and the probable errors of their differ- 
ences. There is practically no disparity between the results for the 
two kinds of audiences. 

It seems that the kind of audience is not very important as regards 
its influence on recall. However, our results may be a function of the 
particular situation, and if that should be the case, other group arrange- 
ments might have different consequences. Therefore no general 
conclusions can be made. From the viewpoint of the organization of 
the audiences further observations and studies would be desirable. 
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Table V contains the data on the learning of the paired associates. 
In all cases learning took place with only the experimenter present. 
The sole difference between the several learning conditions was that, in 
the first, the subjects did not expect an audience, while in the other 
two, they did. Naturally we would expect the learning efficiency to 
be about equal throughout the experiment, or at least not to vary to a 
greater degree than could be explained as due to chance. Yet we find 
that learning was on the average considerably easier when the sub- 
jects did not expect an audience. The difference in trials required to 
learn the series of paired associates when expecting no audience and 
when anticipating an audience is a total of 28 trials or a mean of 1.40 


TaBLe V.—Tue Errect or EXPECTANCY OF AN AUDIENCE FOR RECALL, AT THE 
Time OF LEARNING, ON THE RATE oF LEARNING 














| Total Average | 
Condition | trials to trials per | PE 
| lean | list | 
SEER SC cain rae em Y, Gea ere 96 4.80 | .32 
Inattentive audience.................... 124 6.20 .32 
Attentive audience..................... 123 6.15 34 
Difference between no audience and atten- | 
SE AS REIT Se ee ra 28 1.40 45 
Difference between inattentive and atten- | 
TREE a RT SOE Sa Pe 0 | 0 











trials. The probable error of the difference of the two means is .45. 
The actual disparity is a little over three times its probable error, 
indicating significance. There are no significant differences for the 
two different kinds of audiences. 

The fact that the people needed more trials to learn the material 
when they were expecting an audience may be explained as being due 
to difference in anticipation, and integration with reference to the 
learning situation. Anticipating an audience seemed to have been 
inhibitive to the learning process. The subjects’ reports clearly indi- 
cate that their knowledge of having to recall before spectators caused 
some uneasiness. They appeared to learn with the intention of know- 
ing the words well, and of being able to tell them to somebody. Such 
an attitude was not recognizable with the subjects who did not look 
forward to an audience; these subjects appeared to learn without any 
definite end in view. These are the facts as they are before us. We 
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conclude from them that differences in expected conditions of recall 
may cause corresponding differences in rate of learning. 


CONCLUSIONS 


Restating the facts as they appear from our investigation we find 
that: 

1. If learning takes place privately, the presence of a group at the 
time of recall is detrimental to the efficiency of reproduction. 

2. The kind of audience is not very important as regards its 
influence on recall. However, it may be that in this connection the 
results are a function of the particular situation, and that other group 
arrangements might have different consequences. 

3. The anticipation of an audience for recall, at the time of learn- 
ing, increases the learning time, and, in spite of this, the amount 
remembered is less when recall occurs before an audience. 

These results suggest that the pupil who complains of not being 
able to remember well when requested to recite before a class may not 
be trying to escape responsibility. His inability to recall reveals a 
fundamental factor of memory, and shows that the situation during 
the time of learning and during the time of recall are equally important 
in facilitating or inhibiting the response. 
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THE RELATIVE INFLUENCE OF VISUAL AND 
AUDITORY FACTORS IN SPELLING 
ABILITY 


GEORGE W. HARTMANN 


Department of Psychology, Pennsylvania State College 


A. BACKGROUND OF THE PROBLEM 


The spelling of the average undergraduate is notorious. One 
suspects that the great vogue of objective examinations may be partly 
explained by their power to alleviate eyestrain due to illegible papers, 
disgust caused by childish syntax, and irritation born of repeated 
misspellings. These common deficiencies are especially prominent in 
science courses where the difficulties of mastering a technical vocabulary 
appall even the superior student. How many instructors are there 
who after exposing their charges to a year of its content find them 
spelling psychology more like philosophy, physiology, and philology 
than like its own true self? 

The various explanations which have been advanced for special 
abilities and disabilities as they manifest themselves in academic or 
vocational fields are suggestive but hardly convincing. In the case 
of such a restricted function as spelling, e.g., it is easy to demonstrate 
that it is not an exclusive property of high or low intelligence, although 
it is equally certain that it is not as independent of gereral native 
ability as some extremists would have us believe. An analysis of the 
commonest types of errors leads to the suspicion that various sensory 
and motor idiosyncrasies are the major determiners of performance, 
but which ones are crucial is as yet unknown. The current view seems 
to be that every poor speller must be individually diagnosed and 
specific remedies prescribed—an obvious consequence of an implicit 
belief in plural causation. 

However, the scientific search for adequate generalizations is 
not to be dismissed so lightly. Just as it is profitable to know that 
superior intellects tend to have correspondingly better physiques, 
emotional control, etc., so we ought to know what the significant 
characteristics of a good speller are as distinct from a bad one. 

Among the many explanations advanced for poor spelling, perhaps 
the most recurrent one is that which attributes it to the non-phonetic 


nature of the English language. Were certain sounds uniformly 
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represented by definite letter-forms this alone would facilitate correct 
spelling. The advocates of reformed spelling apparently take this 
seriously, despite the fact that errors occur with relatively equal 
frequency in highly phonetic tongues such as German or Spanish. A 
slightly more plausible variant of this argument is that if a person 
could pronounce words correctly he would have no difficulty with 
their orthography. Unfortunately, this fails to account for the fact 
that many Americans can spell French terms properly but pronounce 
them in a very unorthodox manner; and similarly for the Frenchman 
using English. Moreover, most misspellings are substitutions or 
displacements of letters which by no stretch of imagination can be 
attributed to faulty pronounciation. The definitive elimination of the 
pronunciation theory would seem to be inevitable consequence of the 
finding that deaf-mutes are relatively better in spelling than in other 
school subjects. 

Other viewpoints have emphasized the réle of visual perception. 
Here the commonest opinion is that poor spellers are deficient in 
visual imagery, but the tachistoscopic span of apprehension has also 
been considered as the decisive factor. 

Where such diverse and contradictory opinions exist, a resort to 
experimental attack is indicated. To the writer, the problem phrased 
itself as follows: 

Does excellence of auditory or visual perceptive capacity play the 
dominant réle in spelling performance? Is spelling ability more 
closely associated with those reactions mediated by the ear or with 
those involving the eye? The wording of these questions by implica- 
tion minimizes the influence of motor impulsions as it is assumed that 
chance slips of the pen should occur equally often with both groups; 
and it also sets in the background the influence of such central factors 
as intelligence, type of imagery, association, etc. 


B. Tue Testinc Procepure 


This investigation began by administering to large samples of each 
college class from Freshman to Senior (six hundred thirty-six cases 
in all) a test consisting of fifty selected words appearing in both 
the Thorndike and Horn Word Books. Upon a prepared sheet, the 
subjects printed each word as it was spoken and its meaning illustrated 
by a sample sentence. That the test was a reasonable measure of 
spelling ability is indicated by the reliability coefficient of .84 + .007, 
obtained by the split-half technique. For the purposes of this experi- 
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ment, representative cases chosen from the middle and both tails of 
the distribution were selected for intensive study. Sixty-three 
individuals were thus obtained—twenty-four of the best spellers, 
eighteen of the poorest, and twenty-one average ones centering 
around the median score. 


The following test series was administered individually to each of 
the subjects: 

Test 1. Perceptual Span for Meaningful Material (Reliability 
.74).—Upon the middle of twenty small white cards were printed the 


following unusual words, derived largely from Whipple’s well-known 
information test: 


ageratum simony 

Caedmon synecdoche 
cleistogamous trephine 

gneiss trilobite 

guimpe Weismannism 
hemiptera onomatopoeia 
impetigo sphygmomanometer 
intaglio exophthalmic 
metacarpal Abderhalden 
peneplain La Rouchefoucauld 


One-second exposures of the cards were made tachistoscopically; 
during the intervals (which varied with the writing time) the subject 
recorded the word he had seen. The total number of reproductions 
correct in every detail constituted the score. 

Test 2. Visual Recognition (Reliability .64).—Twenty common 
words printed on cards were exposed to the subject at the rate of about 
one per second with instructions to fixate each item. Immediately 
after the original series had been seen, the cards were shuffled with 
twenty other confusion stimuli. The mixed set was then arranged 
by the subject into two piles—one containing the cards the subject 
hed seen and the other containing those he had not seen. Score = 
2 X number of errors in both piles subtracted from 40. (For the 
exact items see the list in the writer’s appendix to Archives of Psy- 
chology, No. 100.) 

Test3. Silent Reading.—Whipple’s High School and College Form 
B was administered in accordance with the time limits and other 


conditions specified in his directions sheet. Score equals the number 
of correct answers. 
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Test 4. Hidden Word Identification (Reliability .88).—This 
consists in underlining a meaningful English word in a page of pied 
material, the material being embedded in a jumbled nonsense collec- 
tion. A two-minute time limit was imposed. Score equals the 
number of words properly underlined. 

Test 5. Letter-digit Substitution Code (Reliability .56)—A simple 
‘‘home-made”’ code using the ten Arabic figures was devised. Time of 
transcription was measured in seconds. 

Test 6. Auditory Memory Span for Digits (Reliability .73).—The 
fourteen number sets from the corresponding section of Whipple’s 
manual were read to the subject in time with a metronome, and 
recorded by him as soon as heard. Averaging the highest correct 
reproduction in both sets gave the desired measure of span. 

Test 7. Pronunciation (Reliability .75)—The thirty-two words 
forming the pronunciation section of the Cross English Test were used. 
The printed list was shown the subject with instructions to pronounce 
each word as though he were being graded for purity of speech by a 
language teacher. The experimenter checked the correct items by 
listening carefully as the subject proceeded. 

Test 8. Auditory Recognition (Reliability .53)—A list of ten 
ordinary words was read at an even rate to the subject with instruc- 
' tions to fixate them. Immediately after this list had been read, a 
longer list of twenty words containing the ten originals plus ten 
confusion stimuli, was offered. As each word in the test list was 
uttered, the subject said ‘‘yes’’ if he had heard it before and ‘“‘no”’ 
if he had not. 

The reliability coefficients presented for each of the foregoing 
measures were obtained by duplicate forms in the case of Tests 2, 4, 5, 8 
and by matching alternate items in tests 1, 6, 7; for Test 3 adequate 
reliability standards were assumed to have been met by the originator. 


C. REsuULTs 


The mean performance on each of the preceding eight tests was 
separately computed for each group of spellers—the superior, the 
average, and the inferior. ‘Table I below summarizes the essential 
data for all classes. 

Before proceeding to an interpretation of the table, let it be noted 
that if spelling ability be conditioned by excellence in visual perception 
then the ‘‘good” group should reveal a definite superiority in Tests 
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1-5 inclusive, but if excellence in auditory reactions be the decisive 
factor then the “‘good”’ group should be superior in Tests 6-8 inclusive. 
The expected positions of both the normal and poor groups under 
either hypothesis may be readily inferred. 

The object of Table I is to show which tests meet the conditions 
of standard reliability in exposing the significant differences between 
groups representing three degrees of spelling ability. It will be seen 
at once that very few do so. In the visual series, Test 1 alone dis- 
criminates the superior from the average and the average in turn from 
the inferior. Tests 2, 3, and 4 all fall short of the conventional 
requirement. Test 5 oddly enough distinguishes the good from the 
mediocre group, but fails to discriminate between the latter and the 
inferior.? 

In the auditory series, no test possesses discriminatory power 
with the exception of Test 7 which does differentiate the average 
from the poor spellers. 

Apparently Test 1 of the visual group is the only one which can be 
used to segregate the various classes of spelling talent. This test 
contains a group of rare words which the subject copies from immediate 
memory as soon as they have been exposed. According to the evi- 
dence which this affords, speed and accuracy of immediate visual 
perception exert the dominant influence in spelling proficiency. 

An attempt was next made to get at the heart of the problem by 
comparing total standard scores in both the visual and auditory 
batteries for each of the three ability groups. This comparison was 
made as follows: The means and sigmas of all subjects in each test were 
determined, and the mean sigma position of each ability group was 
separately computed for the visual and auditory series. This is 
a most laborious operation, but the following table gives the 
condensation: 


TaBLeE II.—Mgan SD PosiTIoNs or THREE SPELLING ABILITY GROUPS IN VISUAL 
; AND AvupiTory TEsTs 





Visual battery (Tests 1-5 inclusive) | Auditory battery (Tests 6-8 inclusive) 


| 


Superior | Average ’ Inferior Superior Average Inferior 




















55: — .06 — 62 48 04 ~ 69 





1 Since the scores in Test 5 represent times taken, the lowest va!ues designate 
the best performances. 
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The reasoning which is intended to apply is: If the superior (or 
inferior) spellers are such primarily because of high (or low) visual 
perceptual capacity, then their relative performance should be most 
marked in the visual series. As a matter of fact, the order of excel- 
lence is about the same in both batteries. Good spellers, apparently, 
can use their ears proportionately as effectively as their eyes, which 
leaves us uncertain as to which of these senses their prowess should be 
attributed. 

As the method of reliable differences yielded such ambiguous 
evidence, it was necessary to appeal to further analysis in the hope 
that this would reveal the true relations which conceivably had been 
obscured by some kind of compensatory masking. Raw correlations 
between the spelling scores and the eight tests were therefore com- 
puted, and the following coefficients obtained: 


TasB_Le III.—Raw CoRRELATIONS BETWEEN SPELLING ScorRES AND PERCEPTUAL 














TEsTs 
Test No. Description r PE 
1 Meaningful ee ET Ain hip ates ae bk bemen .78 .03 
2 Visual ene Bia. 4s Suen ae wine .39 .07 
3 Silent reading. . = Ply Pe ED hea A Ee 47 .07 
4 Hidden words. . Ma Riis. KR Te .08 
5 Substitution. . ; beak ten nndn ends 41 .07 
6 Auditory memory span.. ee le on whl eee oc Le .08 
7 Pronunciation. . Reiie a on eee re Wee .06 
8 | Auditory recognition... 20... cece eee .43 .07 











This method seems to check the results presented in Table I for 
again the test of perceptual span is predominant. Since the r between 
spelling and the data of Test 1 is attenuated by the imperfect reliabili- 
ties of the respective scores, Spearman’s correction formula was 
applied, resulting in an r of almost .99! It appears that the two tests 
measure much the same function. 

A final attempt was made to build up a regression equation from 
the three most satisfactory visual tests, viz., Tests 1-3, inclusive. 
The raison d’étre of this was that if it could be shown that admittedly 
visual factors possess adequate predictive power for degrees of spelling 
talent, then they must be viewed as causative influences thereof (in 
Hume’s sense). The following calculations are accordingly presented: 





=a =. - 
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Roza) .784 
Zo = 1.892, + .02522 + .1923 (deviation form) 
SD¢est. zo = 4.54. 


It will be noticed that the multiple correlation is in no way greater 
than the coefficient of zero order between the first test of our series 
and the spelling scores. The obvious implication is that this test 
carries almost exclusively the full burden of prediction. 


D. CONCLUSIONS 


The findings presented in the previous section do not readily lend 
themselves to any clear-cut interpretation, but certain provisional 
statements appear to be justified. In the first place, spelling ability 
is no more a function of general visual perception than it is of general 
auditory perception. Nevertheless it does seem to be closely related 
to the special reaction involved in reproducing tachistoscopically- 
exposed stimuli of a meaningful nature. To use the language of 
Gestalt, good spellers perceive total configurations of the verbal 
sort with greater facility than others. Secondly, spelling ability is 
essentially a central function; with normal subjects peripheral factors 
of an optical or retinal nature are inconsequential. Thirdly, while 
the best way to determine spelling talent is still the sample or scale 
method, it would seem possible to use the meaningful perceptual span 
technique as an alternative thereto, since this has the advantage of 
brevity. Fourthly, all the preceding statements should be interpreted 
in the light of the fact that all the data, especially the correlations, 
pertain to individuals chosen from each of the major segments of a 
larger curve. 


.E. SUMMARY 


This study concerned itself with the following question: Is there 
. any form of sensory perception which is the major determiner of 
spelling proficiency? Sixty-three college students, representing respec- 
tively the best, the worst, and the middle levels of performance, were 
given eight laboratory tests singly. These tests fell into two groups— 
one requiring the use of the visual pathways and the other depending 
on the receptors for sound. 

Statistical treatment of the results showed that the visual battery 


as a whole did not discriminate among the groups any better than the 





gel 
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general auditory battery, and vice versa; one visual test, however 
(that of immediate memory span for meaningful visual stimuli), 
correlated .78 + .03 with the spelling criterion and served effectively 
to segregate the three classes of ability. 

The major outcome of this investigation is that spelling ability 
is largely dependent upon one special form of visual reaction and not 


upon general superiority in any sense modality or upon a common 
integrative capacity. 
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FURTHER DATA CONCERNING THE EFFECT OF 
WEIGHTING EXERCISES IN NEW-TYPE 
EXAMINATIONS 


Cc. W. ODELL 
University of Illinois 


The authors of several of the early standardized tests devoted 
considerable labor and care to the determination of supposedly accu- 
rate weights, usually on the basis of difficulty, for the separate elements 
composing their tests. Within a few years, however, a strong tendency 
to discontinue the use of such weights appeared. Studies by Douglass 
and Spencer! and others showed such high coefficients of correlation 
between scores based upon such weights and so-called ‘“‘unweighted’’ 
or raw scores, that is, those based upon one point for each element, 
that they led to the conclusion that the considerable amount of extra 
work involved in deriving and using weights was unnecessary. In a 
recent article Corey? has presented some data which scarcely support 
this conclusion. He had each item of a new-type test in educational 
psychology weighted by six instructors in that subject. From the 
results he determined the correlations between raw scores and those 
computed according to these six series of weights, and also the effects 
of the different weightings upon letter marks into which the test 
scores were transmuted. Five of the six coefficients with the raw 
scores ranged from .82 to .88, the other being .96. Approximately 
one-fourth of the papers were given the same letter mark according 
to all seven scores, and another fourth each according to six, five, and 
four of the seven. The marks according to the six instructors varied 
from those based on raw scores in from twenty-two to forty-nine 
per cent of the cases. 

Because of the considerable disagreement between these results 
and others previously obtained, the present writer was led to make 
two studies along the same line. Before proceeding to present his 
own data, however, he wishes to offer one comment on Corey’s investi- 


| 


1 Douglass, Harl R and Peter L. Spencer: Is It Necessary to Weight Exercises 
in Standard Tests? Journal of Educational Psychology, Vol. XIV, February, 1923, 
pp. 109-112. 

2 Corey, Stephen Maxwell: The Effect of Weighting Exercises in a New Type 
of Examination. Journal of Educational Psychology, Vol. XXI, May, 1930, pp. 
383-385. 
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gation. It does not appear in the article, but the writer happens to 
know that five of the six instructors assigned weights of zero to some 
of the elements, and thus altered the conditions of the experiment. 
A test which contains some items that do not count in determining 
the score is for all practical purposes no longer the same test as if 
all the elements were counted, so that the general effect of this factor 
was to lower the obtained correlations. 

Part of the data which the writer wishes to present are based upon 
a fifty element multiple-answer test with four suggested answers to 
each element that was taken by sixty-two students in educational 
measurements. The elements in the test were weighted upon five 
different bases in addition to the raw score. These bases will be 
referred to by subscripts as follows: 

1 = raw score, one point to each element 

2 = percentage of students answering correctly 

3 = difficulty as determined through application of normal curve to 2 

4 = random distribution of weights one to five 


5 = random distribution of weights one to ten 
6 = second random distribution of weights one to five 


It will readily be seen that the bases of weighting employed in this 
experiment were such as to give no promise of agreement between 
the weights, thus making the conditions of the experiment much 
more unfavorable to securing high correlations between raw and 
weighted scores than if weights had been assigned according to the 
opinions of instructors who might be expected to agree to some extent 
at least. Not only this, but in the case of bases two and three a high 
negative correlation might be expected, since one was based directly 
upon how easy the elements were and the other somewhat less directly 
upon their difficulty. The correlations between the various lists of 
weights are given in Table I. 


TaBLe I.—CoeEFFICIENTS OF CORRELATION BETWEEN WEIGHTS OF ELEMENTS IN 


First Stupy 
fo3 = —.62 Tx, = .03 tf = —.21 rse = .03 
Toe = —.03 T3535 = .24 Te = .07 
Tos = —.14 Ts = 19 


Tse = —.1l 


The writer’s second study was similar except that instead of the 
three random assignments of weights, three instructors actually 
assigned them, and that the weights for ease and difficulty were both 
taken on the same basis so that the correlation between them was 
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—1.00. The test in this case was a twenty-two element single-answer 
test in methods of teaching. The coefficients of correlation between 
the weights in this case are in Table II. 


TaBLe II].—CokrrFrFriciENTs OF CORRELATION BETWEEN WEIGHTS OF ELEMENTS IN 
Srsconp Stupy 


To: = — 1.00 T34 = .08 Ts5 = .75 rs = 04 
to. = .09 T3565 = .07 re = .67 

fo = — .05 Ts = — .09 

Toe = .04 


It will be seen that except in the case of the weightings according 
to ease and difficulty and of those by the three instructors, the correla- 
tions between weights were practically zero. Those given by the 
three instructors correlated positively, in one case decidedly low, and 
in the other two only fairly high. 

After weights had been determined as described, each student’s 
score was computed according to each of the six plans of weighting. 
In the first study there were two ways under each, one by totaling the 
points allowed for all the elements to which he gave the correct answers, 


and the other by means of the approved formula for multiple-answer 
Ww 


tests, score = R — 77’ which in this case becomes score = R — 3 
In this, of course, R stands for the number of right responses, and W 
for the number of wrong responses. In the second study only one 
method, the number correct, was employed. The intercorrelations 
between the scores on each of the six bases are given in Table III. 
In the first half thereof the first coefficients are for scores based upon 
correct responses only, and the second for scores based upon the 
formula. 


TaBLeE IJ].—CorrriciENts OF CORRELATION BETWEEN Scores BasED ON 
DIFFERENT WEIGHTINGS 
First Study 


fic = .98, .97 Te3 = 83, .80 T3464 = .90, .88 'e5 = .96, .96 Ts = .98, .99 
T13 = .92, .92 Tau = .97, .96 T35 = .92, .93 Te = .98, .98 
Ti = .98, .98 Te5 = .96, .96 Ts = .94, .93 
Tis = .99, .99 Tos = .95, 95 
Tie = .99, .99 
Second Study 
Tig = .97 Te: = .75 T3s6 = .94 Tes = .99 Ts = .92 
T13 = .95 Teo = 91 rss = .97 Tug = .98 
Ti4 = .98 res = .93 Ts = .96 
Tis = .97 Tn = .96 
Tig = .98 
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It will be seen that only in the case of the third basis, that is, the 
weights determined according to the normal probability curve as 
applied to difficulty, were the coefficients between the raw scores 
and the weighted scores lower than .97, and that in half of the other 
cases in the first study they are .99, and in all the others in both studies 
.97 or .98. Furthermore, among the various bases of unequal weight- 
ing the only cases in which the coefficients dropped markedly below 
.90 were those in which bases two and three were concerned. Such 
a result would be expected from the fact that there was a fairly high 
negative correlation between the weights given on these bases. Of 
the three between the different instructors in the second study, two 
are decidedly high, .98 and .99, whereas the other one is considerably 
lower, being only .92. 

The comment should probably be inserted that the tendency of 
the correlations and other measures obtained from the results of the 
first study to show greater agreement than those from the second is 
undoubtedly due to the fact that the test employed therein was much 
longer than that used in the second. 

Following Corey’s example, the writer also computed the effect of 
different weights upon marks. This was done in the same way as by 
Corey, considering the 7 per cent highest scores as A’s, the next 24 
per cent as B’s, the next 38 per cent as C’s, the next 24 per cent as 
D’s, and the lowest 7 per cent as E’s. Table IV gives figures showing 
the per cents of the pupils receiving the same letter mark according to 
the various numbers of bases of weighting. 


TaBLE IV.—PERCENTAGES OF Papers RECEIVING SAME Marks ACCORDING TO 
DIFFERENT WEIGHTINGS 











k 
Number of different Percentages given same marks 
bases of weighting First study mien 
6 42, 39 38 
5 36, 32 33 
4 12, 18 15 
3 10, 11 14 











From this table it appears that about 40 per cent of the students would 
have received the same letter mark according to all six bases of weight- 
ing, and about another 35 per cent according to five of the six. In 
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other words, in about three-fourths of the cases would either all, or 
all but one, of the different weightings have given the same mark. 

The percentages of cases in which the marks assigned according 
to the different bases except the first differed from it were also deter- 
mined. These are shown in Table V. 


TaBLe V.—PERCENTAGES OF PaPperRs ON WHiIcH Marks ACCORDING TO DIFFERENT 
WEIGHTINGS VARIED FROM THOSE BasED ON Raw Scores 











Percentage of marks differing from those according to 
Basis of weighting sitio bas 
First study Second study 
2 19, 23 24 
3 26, 29 30 
4 13, 21 | 14 
5 11, 19 18 
6 6, 18 12 











A comparison of the results obtained in the writer’s two experi- 
ments with those of Corey shows that they indicate much less dis- 
crepancy or disagreement between scores assigned according to 
different bases of weighting. In view of the fact that the tests were 
so short, one containing only fifty elements and the other only twenty- 
two, the results are even more significant. To the writer they seem 
to offer very strong support of the conclusion that for new-type tests 
there is so little to be gained by unequally weighting the elements 
that it is not worth the labor of assigning such weights and computing 
scores from them. The longer the test the more fully does this con- 
clusion hold. 
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THE HANDWRITING OF INDIANS 
THOMAS R. GARTH 


University of Denver 


In order to ascertain whether or not there are racial differences in 
native traits we shall have to measure those traits. Let us take the 
simple matter of differences in handwriting between Whites and 
Indians. It is believed that there are peculiarities in handwriting 
which run in families in a race. The question is, do these peculiarities 
run in races? Is Indian handwriting peculiar to itself and different 
from that of Whites? 

Immediately the problem becomes a large one. So many things 
need to be considered. There is the question of legibility and speed. 
Another is the question of esthetic quality. Still another is what has 
been ealled by graphologists “characterological’’ interpretation. Yet 
another is consideration of the diagnosis of the handwriting of the two 
races. In the present study we have been compelled to confine our 
inquiry to the first question, z.e., a Comparison of the Legibility and 
Speed of the Handwriting of Whites and Indians. 


PROBLEM 


We have asked ourselves these questions; 

1. Do these full- and mixed-blood Indians write as legibly as White 
children of a like school grade? 

2. Do they write with the same speed? 

3. Which of the factors under consideration have the greater 
influence on legibility, as age, school grade, degree of White blood? 

4. Which of the above factors has the greatest influence on speed? 

5. Does this study indicate that the Indian race is mentally 
retarded? 


MATERIALS 


The Thorndike Handwriting Scale was used in measuring samples 
of handwriting of Whites and Indians. The rating was done by 
students in classes in educational measurements. A sample was often 
rated by as many as ten individuals, and the average of the ratings 
taken as the measure. The speed was determined by dividing the 
total number of letters by the total time of writing. Because the 
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number of cases of the Whites was so small, we have been compelled 
to resort to the use of Starch’s norms for school grades. 
CoMPOSITION OF THE GROUPS AND THE RESULTS OF MEASUREMENT 


Table I will show the racial and educational composition of the 
groups. There were six hundred three full-blood Indians and one hun- 






























































TaBLeE I 
Grade | 4 5 6 7 8 Total 
Ea ics: ie sais bie cha aval 37 | 64 129 191 182 603 
Mixed-bloods............... 14 35 35 64 42 196 
SRI EA ES aren | 38 58 61 58 45 260 
TaBLeE II 
4 5 6 7 8 
Grade 
Medi- |Quar-| Medi- |Quar-| Medi- |Quar-| Medi- |Quar-| Medi- |Quar- 
an tile an tile an tile an tile an tile 
| 
Group: 
Full-bloods......... 14.6 2.2; 15.8 1.7 | 15.0 1.6 | 15.5 1.3 | 16.6 1.2 
Mixed-bloods....... 16.2 1.2 | 14.7 1.3 | 16.8 1.3 | 16.1 1.2 | 16.7 8 
, SS 10.1 7 | 10.8 -6 | 12.1 .6 | 12.6 5 | 13.6 8 
Legibility: 
Full-bloods......... 8.33 .93) 8.74) 1.01) 8.42 .83) 9.28 | 1.12) 10.36 | 1.29 
Mixed-bloods....... 8.33 .96; 7.81 .89) 8.75 .89; 8.81 .95) 9.91 | 1.10 
De indawsbanenes 8.64 .91) 8.78 .93) 9.88 .87; 9.86 | 1.01} 9.90 .83 
Starch’s norms...... ae” - hine'aa | oS eee ae Be FF Bs ey 10.9 
Full-bloods SN over-| 41 |..... oe eee Mh ihisiss a oo 37 
lap. per cent per cent per cent per cent per cent 
Mixed-bloods overlap} 43 |..... a RS ae Sisven a Mcase 28 
per cent per cent per cent per cent per cent 
Speed: 
Full-bloods......... 41.67 |20.47' 53.85 |13.22) 64.35 12.29) 70.78 |14.12) 69.47 |13.49 
Mixed-bloods....... 45.00 |12.21) 47.72 | 9.78) 57.50 (11.46) 72.5 |13.34) 64.28 |16.50 
_ Ey 33.08 | 8.85) 46.92 10.57) 46.54 |11.17| 53.50 |18.32) 68.13 |15.39 
Starch’s norms...... a. sab hed a” Sewuda a” Snecen a re 83 
Full-bloods SN over- ie Bedwes i a Se I a aa 19 
lap. per cent per cent per cent per cent per cent 
Mixed-blcods overlap| 49 |..... 2S eee mn? Bireeed in EP 25 
per cent per cent per cent per cent per cent 








dred ninety-six mixed-blood Indians found in the United States Indian 
Schools at Chilocco, Oklahoma, and Albuquerque, New Mexico; 
and there were two hundred sixty White children taken from city and 
suburban schools of Colorado. Table II will show the medians of 
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ages and scores for the several grades along with the measures of over- 
lapping, while Table III will show certain correlations found between 
the various factors under consideration, i.e., legibility, speed, age, 
school grade, and degree of blood. 


Only when undertaking to find correlations between degree of 
blood and performance was the mixed-blood group enlarged by some 
Whites and full-blood Indians taken at random. 


TasBie III 
(1) Score (2) Speed (3) Age (4) Grade (5) Degree 
Full-bloods ri2 = .245 + .025 ria= (004 + .027 run = .39 + .023 
ra = —.20 + .03 ra = .32 + .024 


ru = .25 = .025 
Mixzed-bloods ri: = .199 + .043 ris= .027 + .045 run = .42 + .0386 ris = .053 +. 


04 

ra = §=6..09 + .044 ra = .37 + .0388 res = —.05 + .04 
ru = .23 + .042 ras = —.17 + .04 

ros= .006 + .05 


Whites ria = .205 + .04 ris = 86.284 + .0388 rue. 
ra = 44 + .08 ram. 
ra™=. 


8&5 
H HH 
282 


Taste IV.—As MottipLe CorRELATIONS FOR MIXED-BLOODS, WITH AGE 
ELIMINATED 
I. Score legibility (1), grade (2), degree (3). 
R, (23) = .33 Using 21 = bi2.s0%2 + bis.2ts 
a, = .38272 + .035z; 
Weight of school grade = .38. Weight of degree of white blood = .036. 
II. Score speed (1), grade (2), degree (3). R:(23) = .26 


21 = Bir.sXe + dis. ove. a, = .3522. — .0572z; 
Weight of school grade = .352. Weight of degree of White blood = — .057. 
INTERPRETATION 


It will be seen upon examination of Table II that the Indians, 
both full- and mixed-bloods, are much older than the Whites. Fora 
school grade the ages of the full-bloods run on the average three years, 
and the mixed-bloods run four years older than the Whites. The 
range of age for full-bloods is from ten to twenty-three years, for 
mixed-bloods is from ten to twenty-three, and for the Whites is from 
eight to fifteen. However, the median scores of legibility for the 
racial groups are just about the same, 7.e., full-bloods, 9.2; mixed- 
bloods, 8.8; and Whites 9.4. On the other hand the speed of the 
full-bloods is greater than that of either the mixed-bloods or the 
Whites. In turn the speed of the mixed-bloods is greater than that 
of the Whites—these median scores for speed are respectively 66.9, 
57.9, and 50.2 letters per minute. The reader must not conclude 
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that the difference in speed is due to race as appears on the surface. 
This will be discussed later on. 

It will be better to examine Table II for a cnmparison of legibility 
scores and speed scores by grades for the racial groups. It will be 
seen here that performances of the full-bloods are on the whole more 
nearly like the White norms from Starch, (as is indicated by the over- 
lapping) than are those of the mixed-bloods. However, they both 
appear as rather inferior in performance to Starch’s norms. They are 
more nearly like our medians for Whites. That is to say our Whites 
suffer in comparison with the Starch norms. If race made a difference, 
the mixed-bloods ought to show up better upon comparison than the 
full-bloods, but they do not. The same is true for speed. The full- 
blood performances are more nearly like the Starch norms for speed 
than are the mixed-bloods as judged by the overlapping. 

If the reader will refer to Tables III and IV, he will see that the 
greatest correlation is found for all groups between legibility and 
school grade, and speed and school grade with exceptions found in the 
White data. Age is a small factor excepting in the case of the Whites 
where it is found to be significantly correlated with speed (.44) and 
slightly with legibility (.28). For Whites there is the usual high corre- 
lation between age and grade (.92). Oneshould notice the insignificant 
correlation coefficients for degree of blood and Legibility and Speed. 
They are respectively .053 and —.051. Multiple coefficients of correla- 
tion (Table IV) were found between legibility and school grade and 
degree, and speed, school grade and degree for the mixed-blood group. 
These are respectively .33 and .26. These are not very high. They 
were calculated for the purpose of determining the weight that should 
be assigned respectively to school grade and degree of White blood. 
For Legibility it will be seen that the ratio for school grade and degree 
is ten to one, and for speed, about six to one with the one a minus 
quantity. It will be seen that degree of blood is a matter of small 
significance. Elsewhere it has been shown that instead of degree of 
blood having the weight, it seems to have the situation may possibly 
be due to the influence of social] status going with the degree of blood. 
A+ any rate degree is of extremely small weight. 


SUMMARY 


What, then, does this study show? 
1. The full-bloods are on the whole somewhat better in legibility 
than the mixed-bloods, and the Whites are best of all. 
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2. The Indians are no more nor less speedy than Whites excepting 
in the Eighth Grade where the Whites excel. | 

3. The greatest factor in influencing score in legibility and speed 
for the Indians is school grade. Age has little significance, and degree 
of White blood practically no significance if any. 

4. This study does not indicate that Indians are mentally retarded 
at least as this is shown in respect to ability in handwriting. It is 
seen that with proper training Indians may compare well with Whites 
in handwriting. 








A MEASUREMENT OF THE KNOWLEDGE OF 
PSYCHOLOGY BEFORE AND AFTER 
FORMAL TRAINING 


CALVIN HALL 


University of California 


The purpose of this investigation was five-fold. We wished to 
find: (1) The amount of improvement on a comprehensive objective 
examination in elementary psychology before and after taking a 
course in that subject; (2) the relation between the scores on the 
first and second taking of the test; (3) the relation between first and 
second taking of the test and the score on a college aptitude test; (4) 
the relation between gains—that is, score on the last examination 
minus score on the first examination—and college aptitude score; and 
(5) the effect of training on test reliability. 

The first and last examinations were identical and were given on the 
initial and final meetings of the class. The examination consisted of 
one hundred eighteen true-false items based upon Perrin and Klein’s 
“Psychology” the chief textbook used. The Washington College 
Aptitude test was given as a measure of college aptitude. This test is 
made up of the usual “intelligence” test items. The subjects were all 
members of a class in Psychology IA at the University of California 
during the summer of 1930. This course extends over a period of six 
weeks. 

The data are presented in Tables I and II. 


TaBLeE I.—Summary oF PopuLaTions, Means, Sigmas, STANDARD ERRORS OF 
SIgMAs, AND COEFFICIENTS OF VARIATION 








Variable N | Mean | sp_ | SD of | Coefficient of 
Cdist. variation 
_ College aptitude............ 100 | 121.02 | 28.328 
First examination........... 94 72.56 5.868 | +.428 8.087 
Last examination............ 94 92.76 | 6.928 | +.505 7.469 




















The lack of or negligible correlations indicate that little relationship 
exists between the variables we were investigating. Considering the 
corrected coefficients alone, however, certain trends are noticeable. 
Thus, whatever is measured by the Washington College Aptitude test 
has more in common with the first presentation of the psychology 
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examination than with thelast. Thisistobeexpected. “Intelligence” 
should be a more important determinant of information before than 
after formal training because other factors enter into the final examina- 
tion, ¢.g., interest, concentration, habits of study, to lessen the effect of 
“intelligence.’”’ The lack of correlation between “gains” and college 
aptitude score can be interpreted in two ways. Either improvement 
in this specific course is not dependent upon any of the factors that 
determine ‘‘intelligence’”’ test score or the law of diminishing returns 
may operate to enable those doing poorly on the first taking of the 


TaBLeE I].—Summary oF UNCORRECTED AND CORRECTED CORRELATION COEFFI- 
CIENTS AND RELIABILITY COEFFICIENTS WITH PROBABLE Errors. (UNcor- 
RECTED r’s IN PLAIN PRINT, CORRECTED 1r’s BY UNDERSCORING AND 
RELIABILITY r’s IN ITALIC 

















| | 
Variable College First —_— ae 
aptitude /| examination | examination 

College aptitude............ 975 . 293 .152 
+ .003 

First examination........... .183 . 400 .484 
+ .068 + .080 

Last examination........... .128 .261 . 730 
+.066 | +.065 | +.042 

es en aw Lieie 6 aren Te § seacse Fl Sevees | 666 
+ .069 | + .O44 











examination to make greater gains than their more informed class- 
mates. The first and last taking of the examination correlate just 
moderately but probably as well as any two course examinations. 
Certainly the score on our examination before formal training cannot 
be used to predict what score will be made on the same examination 
after training. 

The objection may be raised that we did not validate our examina- 
tion. This objection does not vitally affect our results, however. We 
assumed that we were measuring knowledge of psychology when we 
selected questions from the textbook just as every instructor does when 
he constructs an achievement test for grading purposes. 

Further conclusions that should be noted are the following: 

1. Although the absolute variability on the last examination 
increased over that of the first, the difference between the standard 
deviations is not significant. The relative variability as measured by 
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the coefficient of variation reveals the opposite tendency. The last 
examination was ninety-two per cent as variable as the first examina- 
tion. This difference however is probably not significant. 

2. The reliability coefficient of the last examination increased 
markedly over that of the first examination. This demonstrates the 
faet that the subjects tested as well as the inherent nature of the test 
may contribute to the size of the reliability coefficient. Hence any 
statement of a reliability coefficient for a given test should be accom- 
panied by a description of the subjects. 

3. The use of tests for predicting success in specific school courses is 
of doubtful value if the test is not standardized and possesses a low or 
unknown reliability. Such tests are often used as pedagogical devices 
for selecting students where the class is restricted and the writer on the 
basis of his data condemns this practice. 
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TESTS AND MEASUREMENTS for Teachers 


By Ernest W. Tiecs, Dean, University College, University of Southern California. 
Dr. Tiegs here gives those fundamentals of tests and measurements, a knowl- 
edge of which is necessary for accomplishing effective results in the use of 
tests on the elementary, secondary and college levels. The book discusses 
measurement, the use of tests for definite educational purposes,—the con- 
struction, administration, and interpretation of informational objective 
tests, and the selection and nature of standardized tests. Riverside Tezxt- 
books in Education. $2.40 


DIAGNOSTIC AND REMEDIAL TEACHING 


By Leo J. Brurecxner, Professor of Elementary Education, University of Minne- 

sota, and Ernest O. ME.sy, Associate Fides of Education, Northwestern 

University. 
Emphasis in this book is placed on the techniques in test and remedial work 
that experience has proved can be most effectively employed by the average 
teacher. In addition to the general discussion of the individual pupil and 
educational tests, there are special chapters on diagnostic and remedial 
teaching in Arithmetic, Reading, Language, Spelling, Handwriting, the 
Social Sciences, Character Education, and Health Education. Riverside 
Textbooks in Education. $2.75 
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Important New Texts 


AN INTRODUCTION TO 
THE SCIENTIFIC STUDY OF 
CLASSROOM TEACHING 
By A. S. Barr 


Prof. of Education, University of Wisconsin 
A general survey of the scientific study 
of classroom supervisicn, covering such 
subjects as the background of scientific 
supervision, the objectives of education, 
the search for the causes of poor work, 
data-gathering devices, measurement 
of results, and research. It contains 
many very pertinent criticisms of cur- 
rent supervisory practices, scientific and 
otherwise. $2.50 


THE ART OF 
THE TEACHER 
By P.F. Valentine 


Prof. of Education, San Francisco State 
Teachers College 


In the form of literary essays the author 
develops a unique approach to a philos- 
ophy of education through his concep- 
tion of teaching as an art. A variety 


' of the most recent points of view in fields 


related to education is included. $2.00 


MAKING HOMES 


By Hazel Shultz 


Recent Head of Dept. of Home Economics, 
Univ. of Chicago Laboratory Schools 


A valuable home economics text for gth 
to 11th year courses such as “Home 
Making,” ““The House and Its Care,”’ 
etc. $2.00 


METHODS IN 


EDUCATIONAL RESEARCH 


By Frederick Lamson Whitney 


Dir. Educational Research and Prof. of Educa- 


tion, Colorado State Teachers College 
A text for courses in “Research, and 
Thesis Work,” “Thesis Writing,” and 
“Educational Research.” Its objec- 
tives are to offer students an oppor- 
tunity to develop a conception of the 
scientific method, and to aid the beginner 
in educational research to attain some 
skill in gathering, classifying, and 
interpreting objective material. $2.25 


These Are Appleton Books 


D. APPLETON AND COMPANY 
35 West 32nd Street New York City 























By Witut1am Henry PYLE 


The Psychology of the 
Common Branches 


The book is not only one that can 
be used as a textbook but it is an 
exceptionally good reference book. 
Most of the | books of the past ten 
years on this subject are out of 
date. Here is one that has brought 
all of the fundamental subjects up 
| to 1 Journal of Educational Re- 
searc 


$2.20 plus 10 cents postage. 


WARWICK AND YORK 
Publishers Bautimore, Mp. 
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Back of its Scholarship and 
Accuracy 


That is why The Mer- 
riam-Webster is the 
“Supreme Authority”; 
is the most widely 
used reference work 
in the world; is uni- 
versally accepted in 
the courts, colleges, 
schools, and libraries. 












DICTIONARY 


The “Supreme Authority” 






Every State that has an una- 

bridged dictionary for use its schools 

ee a exclusively The Merriam- 
e 


Nearly 100% of all echoolbooks are based 
on the New International for pronuncie- 


po ea spelling, compounding, and division 


The Callege rota standard of pronunciation, 10 % 
to x ge by the Chicago 
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